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PREFACE 


Project  ECHO  was  an  activity  lasting  from  the  middle  of  1966  to 
the  beginning  of  1969,  sponsored  for  the  most  part  by  the  Advanced 
Research  Projects  Agency  (ARPA),  Project  AGILE,  Department  of  Defense, 

The  research  was  conducted  by  General  Research  Corporation. 

The  initial  effort  was  supported  by  General  Research  Corporation, 
which  asked  Alex  Bavelas,  of  Stanford  University  (working  with  Jose  Fulco 
University  of  California,  Santa  Barbara),  to  explore  the  potential  of  a 
technique,  first  introduced  by  Bavelas,  for  investigating  the  ideology 
of  groups.  The  results  of  this  preliminary  work  were  encouraging. 

The  second  period  of  research  began  in  January  1967 under  the 

direction  of  R.  p.  Barthol,  of  the  University  of  California,  Los  Angeles 

The  project  manager  was  J.  W.  Adams,  and  the  research  team  included 

R.  G.  Bridge  and  M.  Rutherford,  with  T.  W.  Milburn  ,  of  Northwestern 
* 

University,  as  consultant. 

The  third  period  began  in  November  1967  under  the  direction  of 
R.  P.  Barthol.  The  research  team  included  R.  G.  Bridge,  R.  de  Mille, 
and  (as  frequent  consultant)  T.  W.  Milburn;  W.  H.  Himting  and  E.  P. 
Hunting  (for  a  few  months);  assisted  by  J.  F.  Heller,  D.  L.  Rourke, 

C.  C.  Haas,  and  D.  F.  Haas.  Some  other  occasional  collaborators  and 
assistants  are  identified  in  the  report. 

H.  P.  Phillips  of  the  University  of  California,  Berkeley,  was  an 
anthropological  consultant  on  the  culture  of  Thailand. 

Colleagues  who  volunteered  to  help  the  investigators  collect  data 
from  foreign  countries  were:  Ravipan  S.  and  Somchai  R.  (Thailand), 

Now  at  DePaul  University. 


G.  M.  Guthrie  (Philippines),  M.  Ross  and  E.  M.  B.  H.  Ombogodonga  (Kenya), 
and  Lcdr  R.  A  McGonigal,  USN  (Vietnam), 

This  report  covers  the  third  period  of  the  research  with  some 
references  to  findings  made  in  the  second  period.  The  report  is  coordi¬ 
nated  with  three  concurrent  documents  that  are  frequently  cited  in  the 
report  and  that  were  produced  during  the  third  period  of  Project  ECHO, 

The  ECHO  Method  and  the  Study  of  Values  (Milburn,  Barthol, 

* 

and  de  Mille,  1968) 

ECHO  Study  at  the  Pentalith  fracturing  Company  (de  Mille 
and  Barthol,  1969) 

ECHO-Vietnam  Final  Report  (Bridge  and  Heller,  1968) 

The  first  document  places  the  ECHO  method  in  the  theoretical  field  of 
psychological  value  study  and  compares  ECHO  to  other  methods  in  that 
field.  The  second  gives  a  detailed  description  of  an  ECHO  study  in  a 
disguised  industrial  setting.  And  the  third  reports  an  ECHO  study  in 
Vietnam. 

The  name  ECHO  was  chosen  at  a  time  when  the  attention  of  the  investi¬ 
gators  was  focussed  on  the  ECHO  message-session  technique.  The  name 
(which  is  not  an  acronym)  connotes  the  acquisition  of  information  from 
members  of  a  group,  processing  of  that  information,  and  (like  an  echo) 
return  of  the  processed  information  to  other  members  of  the  same  group, 
to  be  judged  by  them  against  information  coming  from  some  other  group. 


* 


Names  and  dates  in  parentheses  indicate  citations;  the  reference  list 
is  located  at  the  end  of  this  volume. 


ABSTRACT 


The  ECHO  method  is  a  way  of  observing,  quantifying,  and  describing 
the  patterns  of  value  and  influence  that  are  felt,  verbally  expressed, 
and  often  acted  on  in  human  society.  Understanding  these  patterns  helps 
us  to  understand,  communicate  with,  and  act  effectively  in  a  particular 
group  or  culture.  Such  patterns  can  be  derived  by  asking  a  few 
general  ECHO  questions  to  which  respondents  give  multiple  answers  about 
what  is  good  and  bad  to  do,  and  who  would  approve  or  disapprove.  The 
heterogeneous  data  that  result  are  classified  into  empirically  inherent, 
rather  than  into  rationally  predetermined,  categories.  The  data  are 
classified  by  members  of  the  surveyed  group  as  well  as  by  the  researchers, 
and  the  subsequent  data  analysis  is  mostly  computerized. 

From  the  middle  of  1966  to  the  beginning  of  1969,  Project  ECHO 
achieved  six  main  objectives: 

1.  The  method  was  developed  to  a  point  where  ECHO  could 
give  a  detailed  description  of  the  value  and  influence 
pattern  of  a  group,  discriminating  in  detail  and  with 
confidence  between  different  groups  and  subgroups  in 
the  same  or  different  cultures. 

2.  The  method's  relevance  to  theory  was  shown. 

3.  ECHO  method  was  found  to  be  reliable  and  concurrently 
valid. 

4.  The  method  was  applied  successfully  in  areas  of  foreign 
language  and  culture,  as  well  as  in  industrial  and 
educational  settings. 

5.  A  detailed  description  of  the  method  was  prepared. 

6.  Special  elements  of  the  ECHO  method  were  prepared  for 
application  in  Thailand. 
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I. 


SUMMARY 


The  ECHO  method  is  a  way  of  observing,  quantifying,  and  describing 
the  patterns  of  value  and  influence  that  are  felt,  verbally  expressed, 
and  often  acted  on  in  human  society.  Understanding  these  patterns  helps 
us  to  understand,  communicate  with,  and  be  effective  in  a  particular 
group  or  culture. 

In  the  ECHO  method,  respondents  from  a  selected  group  give  multiple 
answers  to  a  few  very  general  questions,  such  as  questions  about  what  is 
good  or  bad  to  do,  and  who  would  approve  or  disapprove  of  each  thing. 

The  many  and  varied  responses  are  classified  into  categories  that  are 
inherent  in  the  data,  rather  than  into  categories  predetermined  by  the 
investigator.  Often,  the  responses  are  classified  by  members  of  the 
respondent  group  as  well  as  by  researchers,  so  that  an  exact  and  undis¬ 
torted  picture  can  emerge.  Analysis  of  the  classified  responses  requires 
computer  support,  to  be  economical  and  timely. 

The  main  objectives  of  Project  ECHO  were: 

1.  Develop  the  ECHO  method  to  the  point  where  it  could  give 
a  detailed  description  of  values  and  influences  in  a 
group  and  could  discriminate  with  confidence  between 
different  groups. 

2.  Show  the  relevance  of  the  ECHO  method  to  a  theoretical 
background. 

3.  Evaluate  the  ECHO  method. 

4.  Explore  the  applicability  of  the  ECHO  method. 

5.  Describe  the  ECHO  method  in  detail. 

6.  Prepare  special  elements  of  the  ECHO  method  for  appli¬ 
cation  in  Northeast  Thailand. 

During  the  project,  the  ECHO  method  was  developed  from  a  few  rudi¬ 
mentary  steps  to  a  set  of  21  elements  with  much  greater  analytical  power. 
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At  first,  ECHO  had  quite  limited  ability  to  discriminate  between  different 
groups,  and  required  a  comparatively  large  expenditure  of  time  for  data 
reduction.  With  the  development  of  the  set  of  21  elements  (most  of  them 
steps),  the  investigator  had  many  choices  of  research  design,  even  though 
the  time  required  to  perform  ECHO  operations  had  been  reduced.  The  ECHO 
method  had  attained  the  power  to  not  only  describe  in  detail  the  value 
and  influence  pattern  of  a  group,  but  also  to  discriminate  in  detail  and 
with  confidence  between  groups  and  subgroups  in  the  same  or  different 
cultures.  This  report  presents  the  ECHO  method,  as  it  stood  at  the  end 
of  the  project.  Detail  is  sufficient  to  permit  application  by  scientifi¬ 
cally  trained  investigators. 

The  report  and  its  coordinated  document  The  ECHO  Method  and  the 
Study  of  Values  show  that  the  ECHO  method  is  fully  relevant  to  the  theo¬ 
retical  field  of  psychological  value  study.  These  documents  evaluate  the 
method  in  comparison  with  five  other  methods  of  value  study.  Reliability 
is  found  satisfactory  for  theoretical  and  practical  applications;  and 
concurrent  validity  is  evidenced.  Utility  (cost-benefit  characteristic) 
is  judged  to  be  comparatively  high;  and  informational  utility  (production 
of  unexpected  useful  information)  is  shown,  in  two  examples,  to  be  high. 

The  ECHO  method  was  found  to  be  applicable  in  foreign  languages 
(Spanish,  Thai,  Vietnamese),  in  foreign  cultures  (Thailand,  Kenya, 
Philippines,  Vietnam),  in  industrial  settings  (four  companies;  for 
examples,  see  Barthol  and  Bridge,  1968,  and  de  Mille  and  Barthol,  1969), 
and  in  educational  settings  (many  samples,  third  grade  to  graduate  study). 
The  ECHO  method  was  judged  fruitful  for  the  theoretical  study  of  values 
in  human  behavior. 

Specific  preparations  for  research  in  Thailand  included  a  set  of 
ECHO  questions  that  were  pre-tested  on  five  samples  of  Thai  students  and 
Thai  college  graduates  and  found  satisfactory.  In  the  Thai  language, 

"good  to  do"  and  "bad  to  do"  were  rendered  by  four  questions  meaning 
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"  "not  like  to  do,"  "ought  to  do,"  "ought  not  to  do,"  because 


"like  to  do, 

the  Thai  ^  has  a  much  stronger  moral  connotation  than  the  English  good* 
Thai  instructions  for  ECHO  question  and  classification  sessions  were  also 
prepared,  tested,  and  found  satisfactory. 

The  ECHO  method  is  recommended  for  theoretical  work,  cultural  des¬ 
cription,  and  intergroup  discrimination.  ECHO  is  suggested  as  probably 
appropriate  for  behavior  prediction,  the  enhancement  and  evaluation  of 
training  in  culture-relevant  fields,  the  construction  or  enhancement  of 
informative  or  persuasive  communications,  and  the  development  of  polling 
questions.  The  ECHO  method  is  specifically  recommended  for  research  in 
Thailand  and  in  domestic  problems  of  community  relations.  Recommendations 
for  further  development  of  the  method  are  listed. 

Numerous  incidental  descriptive  findings  and  some  incidental  theo¬ 
retical  findings  are  reported. 
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II.  INTRODUCTION 


A.  PURPOSE  OF  THE  ECHO  METHOD 


1.  Measurement  of  Human  Values 

The  ECHO  method  is  primarily  a  way  of  measuring  human  values. 
Underlying  the  method  is  an  assumption  so  broad  and  basic  that  one  might 
easily  forget  to  make  it  explicit:  the  assumption  that  there  is  a  uni¬ 
versal  human  tendency,  common  to  all  places  and  cultures,  to  exhibit  in 
both  verbal  and  non-verbal  behavior  some  preferences  and  aversions,  some 
obligations  and  prohibitions,  some  hopes  and  fears,  some  satisfactions 

and  disappointments.  In  short,  people  ever3^here  are  assumed  to  feel, 

* 

to  express,  and  to  act  out  their  concepts  of  good  and  bad. 


A  second  assumption  of  the  method  is  that  values  are  held  in  com¬ 
mon  in  homogeneous  groups  of  people.  If  we  ask  a  hundred  people,  "What 

is  a  good  thing  to  do?"  and  eighty  of  them  answer,  "It  is  good  to  help 

others,"  we  assume  that  the  remaining  twenty  also  believe  that  it  is 

good  to  help  others  but  did  not  think  of  mentioning  it  on  that  particular 

day.  The  value  category  "helping  others"  would  then  be  assigned  an 
importance  of  80  percent  for  that  group,  which  indicates  a  strong  value. 


In  this  report,  the  term  value  often  includes  the  ideas  of  both  good 
and  bad,  but  occasionally  good  and  bad  are  distinguished  as  values 
and  disvalues.  The  term  value  may  refer  to  some  feeling  we  ascribe  to 
a  person  (for  example,  we  say  that  he  values  his  life),  or  value  may 
refer  to  something  he  says  (such  as,  "It  is  good  always  to  tell  the 
truth").  The  most  common  use  of  the  term  in  this  report  is  to  desig¬ 
nate  certain  kinds  of  things  that  people  say  are  good  and  bad;  these 
kinds  of  things — more  exactly  described — are  the  titles  given  to  cate¬ 
gories  of  ECHO  responses.  We  may  say,  then,  that  killing  is  a  dis- 
value  in  many  groups — by  which  we  mean  that  in  many  groups  some  answers 
to  ECHO  questions  (such  as,  "it  is  bad  to  kill,"  "murder  is  bad")  have 
been  grouped  together  and  labelled  Killing.  The  term  value  and  related 
terms  (attitude,  norm,  belief,  habit,  custom,  opinion)  are  defined  and 
discussed  by  Milburn,  Barthol,  and  de  Mille  (1968).  Some  other  special 
terms  may  be  found  in  the  Glossary,  Appendix  I. 
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If  five  people  answer,  *'lt  is  good  to  take  a  vacation,"  we  assume  that 

the  remaining  ninety-five  also  believe  it  is  good  to  take  a  vacation, 

* 

but  did  not  think  of  mentioning  it.  The  value  category  "take  a  vacation" 
would  be  assigned  an  importance  of  five  percent,  which  indicates  a  rela¬ 
tively  weak  value,  but  one  still  important  enough  to  be  mentioned  spon¬ 
taneously  by  members  of  that  group. 


2.  Measurement  of  Influences 

In  addition  to  measuring  values,  the  ECHO  method  measures  the 
influences  that  people  associate  with  values.  Many  values,  for  example, 
involve  social  obligations  or  prohibitions.  People  say  that  it  is  a 
good  thing  to  be  polite  or  a  bad  thing  to  steal.  When  we  ask  who  would 
approve  of  being  polite  or  who  would  disapprove  of  stealing,  one  respon¬ 
dent  may  tell  us  that  his  parents  would  approve  of  being  polite  and  the 
police  would  disapprove  of  stealing;  another  respondent  may  say  that  he 
himself  would  both  approve  of  being  polite  and  disapprove  of  stealing. 
These  approving  or  disapproving  figures  are  called  sources ,  because  they 
are  seen  as  giving  approval  or  disapproval  for  good  or  bad  behavior; 
they  are  sources  of  one  kind  of  influence. 

Another  kind  of  influence  is  touched  on  by  questions  that  ask, 
"What  is  a  good  thing  to  happen?  Who  or  what  would  cause  it?"  Patterns 
of  cause  and  effect  are  quite  different  from  patterns  of  approval  for 
behavior,  but  both  are  patterns  of  value  and  influence. 


As  with  values,  the  tendency  to  be  aware  of  social  approval  and 
disapproval  and  the  tendency  to  see  cause  and  effect  relationships  are 


The  assumption  about  common  values  was  generally  applied  in  Project 
ECHO  and  is  further  discussed  on  pages  60  to  62  ,  Occasionally,  the 
assumption  may  be  contradicted  by  the  data,  as  in  the  Pentalith  Study 
(de  Mille  and  Barthol,  1969),  where  two  small,  mutually  contradictory 
categories  were  interpreted  as  reflecting  the  differing  views  of  two 
previously  unsuspected  subgroups. 
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assumed  to  be  universal  in  human  society.  The  sources  mentioned  by 
members  of  a  group  are  also  assumed  to  be  held  in  common  by  that  group, 
the  more  important  sources  being  those  mentioned  by  more  people, 

3 .  Values  and  Behavior 

The  main  reason  for  measuring  patterns  of  value  and  influence  is 

that  human  behavior  is  largely  motivated  by  such  patterns.  People  try 

to  do  the  things  they  like  to  do  or  that  are  approved  in  society;  they 

* 

try  not  to  do  things  they  dislike  or  that  are  disapproved.  The  balance 
between  private  preferences  and  public  morality  may  change  often,  so 
that  a  person  may  conform  to  social  standards  at  one  time  and  violate 
them  at  another.  However,  at  any  particular  moment,  he  will  feel  and 
act  out  a  particular  set  of  values  and  expectations  of  influence.  These 
forces  combine  with  other  factors  (such  as  information)  to  shape  his 
behavior  from  moment  to  moment. 

Since  ECHO  produces  detailed  descriptions  of  the  value  and  influ¬ 
ence  patterns  felt,  expressed,  and  to  a  considerable  degree  acted  out  by 
groups  of  people,  it  tells  us  many  things  we  want  to  know  about  those 
groups;  it  helps  us  to  understand  things  that  may  be  inexplicable  with¬ 
out  such  information.  Why,  for  example,  are  there  so  many  hungry  people 
in  India  when  there  appears  also  to  be  an  oversupply  of  beef?  The  expla¬ 
nation  can  be  given  almost  entirely  in  terms  of  a  value:  in  the  Hindu 
culture,  it  is  a  bad  thing  to  slaughter  a  cow.  Who  would  disapprove? 

Holy  men,  wise  men,  or  the  gods.  Values  act  in  daily  life  in  all  cul¬ 
tures  to  shape  behavior;  understanding  the  values  of  a  culture  is  indis¬ 
pensable  for  understanding  the  society,  moving  through  it  successfully, 
or  communicating  effectively  with  its  members. 


Approval  or  disapproval  may  stem  from  many  social  groups,  institutions, 
or  subcultures;  no  specific  social  frame  of  reference  is  assumed  in 
this  discussion. 
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B.  BRIEF  DESCRIPTION  OF  THE  ECHO  METHOD 

1.  The  ECHO  Questions 

The  usual  technique  for  collecting  ECHO  data  is  quite  simple. 

Members  of  the  group  to  be  surveyed  are  brought  together  and  given  packets 

of  data  processing  cards  called  question  cards.  Each  card  has  a  set  of 

* 

two  printed  questions,  and  the  respondent,  or  ^  ,  writes  an  answer  to 
each  question.  Cards  that  have  already  been  used  by  the  ^s  are  called 
"response"  cards."  The  first  response  card  illustrated  in  Fig.  1  is  from 
an  American  high  school  girl,  who  says  that  a  good  thing  to  do  is  to  "be 


r 

WHAT  IS  A  GOOD  THING  TO  DO?  _ Wa 

_ _ Wi  ^u-k.  , 


to 

Oi 

cv 

to 

I 


WHO  WOULD  APPROVE? 


□ 


GO  ON  TO  THE  NEXT  CARD  IN  THIS  PACKET 


0 


r 


I  QUE  SERI'A  algo  HALO  QUE  UD.  PUDIERA  HACER? 


l  QUIEN  LO  DESAPROBARIA? 

CONTINUE  EN  LA  PROXIMA  TARJETA  INCLUIDA  EN  ESTE  PAQUETE 


O 


Figure  1.  ECHO  Reponse  Cards 


In  this  report,  ^s  will  stand  for  respondents  to  the  questions  or  sub¬ 
jects  in  an  experiment.  The  ^  will  stand  for  the  experimenter,  investi¬ 
gator,  or  session  administrator. 
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more  useful  around  the  house.  Help  with  more  of  the  housework,"  and  that 
her  parents  would  approve.  The  second  card  is  from  a  male  Cuban  refugee, 
who  says  that  a  bad  thing  is  to  "abuse  some  poor  devil  who  hasn’t  done 
anything  wrong,"  and  that  he  himself  would  disapprove. 


Because  ^  wishes  to  find  out  about  many  values,  each  ^  is  given 
several  question  cards  (usually  7  or  10)  bearing  the  same  set  of  ques¬ 
tions  (for  example,  "What  is  a  bad  thing  to  do?  Who  would  disapprove?") 

The  ^s  are  instructed  to  write  something  different  on  each  card.  In  20 
to  35  minutes  the  ^s  have  completed  their  good  and  bad  cards  and  any 

supplementary  cards  that  may  be  in  the  packet.  The  cards  are  collected  ' 

and  the  question  session  is  over. 

2 .  ECHO  Classification 

The  next  step  is  to  classify  the  responses.  Classification  is  , 

necessary  because  ^  cannot  know  in  advance  what  kinds  of  answers  the  ^s 

i 

will  give.  American  college  students  may  say,  among  many  other  things,  . 

i 

that  it  is  good  to  protest  the  ills  of  society,  to  make  friends,  and  to  I 

I 

know  and  be  true  to  yourself.  Kenyan  students  may  say  that  it  is  good  to 
be  loyal  to  your  country,  to  do  something  in  agriculture,  and  to  obey 
your  parents.  Thai  students  may  say  that  it  is  good  to  be  polite,  to  be 
generous,  and  to  do  things  that  are  fun. 

When  ^s  are  not  directed  to  answer  specific  questions  (such  as, 

"Is  it  better  to  go  to  college  or  get  a  job?")  but  are  permitted  to 
respond  as  they  please  to  very  general  questions  (such  as,  "What  is  a 
good  thing  to  do?") ,  and  when  they  are  required  to  answer  the  same 
question  several  times,  the  result  is  a  mass  of  heterogeneous  data  that 
cannot  be  fitted  into  categories  prepared  ahead  of  time  by  Instead, 

categories  must  be  formed  out  of  the  responses  themselves,  so  that  the 
inherent  meaning  of  the  data  can  emerge. 

See  footnote  on  previous  page. 

i 
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The  ECHO  classification  technique  is  the  means  of  finding  the  cate¬ 
gories  of  good  and  bad  things  that  are  inherent  in  the  data.  Three  people 
work,  first  independently  and  then  as  a  team  of  classifiers,  grouping 
together  cards  (responses)  that  have  the  same  meaning  to  them.  When  the 
team  has  finished  grouping  the  cards,  all  of  the  responses  have  been 
classified  into  a  system  of  categories;  each  category  containing  responses 
that  mention  the  same  kind  of  good  (or  bad)  thing.  The  classifiers  then 
make  up,  for  each  category,  a  title  that  summarizes  all  of  the  responses 
in  the  category. 

This  classification  technique  assures  that  the  respondents’  value 
system  (or  hierarchy,  since  some  values  are  mentioned  by  many  ^s  while 
others  are  mentioned  by  few)  will  emerge  undistorted  by  the  preconceptions 
of  the  investigator.  Sometimes,  however,  the  classifiers  themselves  may 
have  inappropriate  preconceptions  and  may  group  cards  in  a  way  that  does 
not  quite  fit  the  indigenous  value  system  of  the  group.  Inappropriate 
grouping  is  particularly  likely  to  happen  when  the  classifiers  are  not 
familiar  with  the  culture  (as  for  example,  American  research  assistants 
who  classify  the  responses  of  Philippine  or  Thai  respondents) .  To  elimi¬ 
nate  any  distortion  that  might  be  introduced  by  staff  classifiers,  the 
responses  are  often  classified  by  members  of  the  group  that  is  being 
surveyed.  These  indigenous  classifiers,  being  familiar  with  the  values 
of  their  own  group,  with  the  social  background  of  those  values,  and  with 
the  language  in  which  the  responses  are  given,  may  produce  some  categories 
that  are  quite  different  from  those  produced  by  nonindigenous  classifiers. 
On  the  other  hand,  staff  classifiers  usually  work  more  accurately  and 
produce  categories  that  are  technically  better.  The  two  kinds  of  classi¬ 
fication  are  applied  to  the  same  data  and  are  mutually  corrective,  so 
that  the  end  result  is  a  well-constructed  and  undistorted  hierarchy  of 
values . 

3.  ECHO  Data  Analysis 

The  analysis  of  ECHO  data  depends  on  automated  procedures,  on  the 
use  of  data  processing  machines  and  computers.  Of  course,  all  operations 
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could  be  accomplished  by  hand,  but  the  cost  (especially  in  time)  would 
be  prohibitive  for  most  applications.  The  mass  of  data  is  so  large  and 
complex  that  practical  use  of  the  method  requires  automation  of  all  but 
the  few  judgmental  steps. 

The  collection  of  ECHO  answers  on  data  processing  cards  is  the 
first  step  in  the  automated  analysis.  Each  card  is  prepunched  with 
identification  codes  that  make  it  easy  to  tell  one  ^  from  another  and 
one  group  or  subgroup  from  the  next.  (Though  each  ^  may  be  completely 
anonymous,  E  must  still  distinguish  between  ^s.) 

After  the  classification  codes  have  been  punched,  the  data  cards 
are  machine  processed  through  a  series  of  steps.  The  computer  outputs 
include  lists  of  value  hierachies  and  of  source  hierarchies,  tables  that 
compare  groups,  lists  of  classified  responses,  and  other  kinds  of  results 
that  are  ready  for  interpretation  by 

4.  Difference  Between  ECHO  and  Polling  Techniques 

The  ECHO  method  is  not  considered  to  be  a  replacement  for  the 
familiar  polling  techniques  in  which  respondents  answer  specific  ques” 
tions  (whether  few  or  many)  and  answer  each  question  only  once.  The  two 
kinds  of  techniques  produce  different  information.  Specific  poll  ques¬ 
tions  can  provide  very  accurate  measurement  of  the  things  the  pollster 
is  interested  in.  When  the  pollster  has  chosen  his  questions  well,  the 
information  may  be  very  useful.  But  in  order  to  choose  his  questions 
well,  the  pollster  must  be  very  familiar  with  the  culture  and  population 
to  which  those  questions  relate.  The  remarkable  success  that  polls  have 
achieved,  for  example,  in  predicting  American  election  outcomes  depends 
in  part  on  the  fact  that  the  voting  ritual  is  wholly  familiar  and  well- 
understood.  How  well  might  pollsters  do  in  predicting  next  year’s  migra¬ 
tions  of  tribesmen  in  Kurdistan?  Not  nearly  so  well,  one  supposes,  and 
one  of  the  main  reasons  is  that  they  would  not  know  what  questions  to 
ask. 
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The  pollster  must  have  prior  knowledge  about  values  that  are  impor¬ 
tant  to  the  group  being  questioned.  Having  such  knowledge,  the  pollster 
can  ask  the  right  questions.  The  main  purpose  of  ECHO  is  to  find  out  just 
what  important,  what  values  are  expressed  by  most  members  of  a  partic¬ 
ular  group. 

Polls  can  get  answers  to  questions  that  the  respondents  are  not 
really  interested  in.  If  this  happens,  the  results  are  misleading.  If 
asked,  75  percent  of  Americans  might  say,  for  example,  that  there  should 
be  no  abstract  paintings  on  the  walls  of  the  White  House.  How  important 
is  such  a  preference?  We  cannot  know  whether  such  a  question  should  even 
be  asked  unless  we  examine  the  prevailing  hierarchy  of  values  about  the 
conduct  of  the  President.  The  ECHO  method  is  most  useful  in  determining 
values  and  value  hierachies  about  which  we  have  little  prior  knowledge. 

C.  ORIGINS  OF  THE  ECHO  METHOD* 

The  use  of  broad,  open-ended  value  questions  can  be  traced  back  to 
Osborn  (1894) ,  who  asked  children  what  they  should  do  to  be  called  good 
or  bad.  Osborn  classified  the  children's  responses  into  several  kinds 
of  good  or  bad  behavior  and  found  that  more  than  half  of  the  group  men¬ 
tioned  obedience  while  only  about  a  quarter  mentioned  truthfulness.  He 
made  comparisons  between  boys  and  girls  and  between  classes  from  two 
different  schools. 


Bavelas  (1942)  introduced  a  technique  that  was  the  direct  fore¬ 
runner  of  the  ECHO  method;  he  described  its  application  to  school  chil¬ 
dren  as  follows : 


The  first  question  was,  'What  could  a  child  of  your  age 
do  at  school  that  would  be  a  good  thing  to  do  and  someone 
would  praise  him?'  When  the  child  had  answered  the  question, 
the  experimenter  asked,  'Who  would  praise  him?'  This  unit 
was  repeated  three  times,  the  second  and  third  time  the  child 
being  asked,  'What  else  could  a  child  of  your  age  do  and  some¬ 
one  would  praise  him?'  On  the  fourth  repetition  the  experi¬ 
menter  changed  the  statement  by  inserting  the  adjective  'very' 


A  more  detailed  discussion  is  given  by  Milburn,  Barthol,  and 
de  Mille  (1968). 
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before  the  word  ’good,’  and  by  changing  the  second  half  of 
the  question  to  read,  ’Who  would  praise  him  very  much?’ 

On  the  seventh  repetition  the  experimenter  read  two  ’very’s’ 
where  one  had  been.  In  all,  there  were  nine  repetitions  of 
the  question  asking  for  praisable  activity.  This  unit  of 
praise  questions  was  followed  by  a  unit  of  nine  scold  ques¬ 
tions  phrased  in  a  corresponding  manner.  [1942,  p.  373] 


The  Bavelas  technique  was  superior  to  Osborn’s  procedure  in  three 
important  ways: 

1.  Bavelas  presented  each  question  a  standard  number  of  times 
and  elicited  multiple  responses  from  every  subject. 

2.  The  source  of  praise  or  blame,  implicit  in  Osborn’s 
questions,  was  made  explicit  by  Bavelas. 

3.  Relationships  between  valued  or  disvalued  acts  and  the 
sources  of  social  reinforcement  were  analyzed. 

Kalhorn  (1944)  applied  the  Bavelas  technique  in  a  study  of  Menno- 
nite  and  non-Mennonite  school  children,  finding  intergroup  differences 
in  values  and  in  sources  of  approval /dis approval .  Havighurst  and 
Neugarten  (1955)  used  the  technique  to  compare  midwestern  white  children 
with  children  of  ten  American  Indian  communities. 


The  infrequent  use  of  this  promising  technique  during  the  24  years 
between  its  invention  and  the  beginning  of  Project  ECHO  can  probably  be 
laid  chiefly  to  the  difficulty  of  analyzing  the  data  without  suitable 
automated  procedures. 
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III.  RESEARCH  OBJECTIVES 


The  main  objectives  of  Project  ECHO  were  the  following: 

1.  Develop  the  method  to  the  point  where  ECHO  would  be 
capable  of: 

a.  Describing  in  detail  the  value  and  influence  pattern 
of  a  selected  group. 

b.  Discriminating  in  detail  (and  with  confidence)  be¬ 
tween  groups  and  subgroups. 

2.  Define  the  relevance  of  the  ECHO  method  to  the  theoretical 
field  of  psychological  value  study. 

3.  Evaluate  the  ECHO  method  against  criteria  of  reliability, 
validity ,  utility ,  and  informational  utility,  in  comparison 
with  other  methods  of  value  study. 

4.  Explore  the  applicability  of  the  method  to  populations 
speaking  other  languages,  to  foreign  cultures,  to  indus¬ 
trial  and  educational  settings,  and  to  other  groups. 

5.  Articulate  and  record  the  elements  of  the  method  so  that 
ECHO  could  readily  be  applied  (with  suitable  modification) 
by  a  scientifically  trained  investigator. 

6.  Prepare  specific  ECHO  elements  for  application  in  North¬ 
east  Thailand. 
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IV.  RESULTS  AND  DISCUSSION 

The  long  duration  of  Project  ECHO  (2-1/2  years),  the  number  of 
methodological  problems  that  were  solved,  and  the  variety  of  available 
data  samples  made  the  outcomes  numerous,  complex,  and  varied.  This 
chapter  is  divided  into  seven  sections  (A  to  G)  in  which  the  results  of 
Project  ECHO  are  reported  and  discussed.  Most  of  the  results  were  ob¬ 
tained  during  the  third  period  of  research  (which  began  in  November, 
1967),  but  several  important  findings  from  the  second  period,  reported 
earlier  by  Barthol  and  Bridge  (1967)  ,  are  reviewed  here. 

Section  A  describes  the  ECHO  method  as  it  stood  at  the  end  of  the 
project.  Twenty-one  elements  of  the  method,  most  of  them  procedural 
steps,  are  listed  and  briefly  explained;  eighteen  of  the  elements  are 
presented  in  a  diagram  (Fig.  2). 

Sections  B  and  C  give  details  of  two  unique  ECHO  elements :  the 
ECHO  classification  technique,  and  the  ECHO  sources. 

Section  D  is  a  further  elucidation  of  the  method.  Parameters  and 
known  requirements  or  limitations  of  the  method  are  discussed  at  length. 
Particularly  important  among  the  parameters  are  the  six  variables  con¬ 
tained  in  the  ECHO  questions:  role,  event,  valuation,  reinforcement  or 
agency,  source,  and  additional  context. 

Section  E  reports  findings  about  the  applicability  of  the  ECHO 
method  to  populations  that  speak  other  languages,  to  foreign  and  esoteric 
cultures,  to  hostile  domestic  groups,  and  to  work  groups.  The  problem 
of  measuring  value  change  is  discussed,  and  the  ability  of  ECHO  to  dis¬ 
criminate  between  groups  and  subgroups  is  described. 

Section  F  gives  an  evaluation  of  the  ECHO  method  against  the  cri¬ 
teria  of  reliability,  validity,  utility,  and  informational  utility.  The 
discussion  is  coordinated  with,  and  adds  some  detail  to,  the  evaluation 
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of  ECHO  given  in  The  ECHO  Method  and  the  Study  of  Values  (Milburn, 
Barthol,  and  de  Mille,  1968). 

Section  G  subsumes  the  theoretical  groundwork  of  ECHO,  given  in 
The  ECHO  Method  and  the  Study  of  Values;  some  specific  theoretical  find¬ 
ings  are  reported. 

A.  ECHO  METHOD,  1968 

At  the  end  of  the  ECHO  Project,  the  ECHO  method  had  been  developed 
well  beyond  the  rudimentary  steps  described  by  Barthol  and  Bridge  (1967). 
Figure  2  shows  the  1968  ECHO  procedures  from  the  point  where  data  are 
first  collected  to  the  point  where  the  findings  are  used;  the  heavy 
arrows  indicate  a  sequence  of  procedures  that  are  used  in  most  studies; 
the  lighter  arrows  indicate  various  options  that  can  be  used  as  needed. 

The  numbered  headings  in  the  following  paragraphs  take  up  different 
ECHO  procedures  (or  products),  of  which  items  4  to  21  are  represented  in 
Fig.  2.  The  first  fifteen  headings  deal  with  the  main  procedures;  the 
last  six,  with  optional  procedures.  The  descriptions  of  the  procedures 
will  be  brief;  extended  discussions  of  some  items  will  be  deferred. 

1.  Statement  of  Research  Problem 

ECHO  studies  may  have  theoretical  or  applied  purposes,  or  both. 

If  theoretical,  the  study  may  test  hypotheses  or  be  merely  exploratory. 

If  applied,  the  study  may  generate  information  of  some  particular  kind 
for  practical  use  at  a  specified  time.  Such  considerations  will  largely 
control  the  design  of  the  study.  The  ^*s  first  step  is  to  state  the 
research  problem. 

2 .  Design  of  Study 

The  designing  of  behavioral  or  social  research  implies  a  background 
of  scientific  education,  training,  and  experience  that  cannot  be  imparted 
in  a  list  of  procedures.  A  review  of  the  theoretical  background  of  ECHO 
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Figure  2.  The  ECHO  Method,  1968 
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is  given  by  Milburn,  Barthol,  and  de  Mille  (1968).  Many  of  the  method¬ 
ological  strictures  that  apply  to  ECHO  studies  are  among  those  applicable 
to  psychological  testing  (Anastasi,  1961).  Nevertheless,  some  selected 
questions  of  design,  most  of  them  peculiar  to  the  ECHO  method,  may  be 
usefully  specified. 

a.  Selection  of  Population(s) 

The  _E  must  answer  the  question:  "What  kind  of  people  are  we  inter¬ 
ested  in,  and  how  do  we  define  them  as  a  population?"  In  the  Intercultural 
College  Study,  for  example  (see  Appendix  II),  ^  defined  the  populations 
as  college  students  in  different  countries. 

b.  Selection  of  Sample(s) 

The  _E  must  select  groups  that  will  be  defensible  samples  of  the 
defined  populations.  In  the  Intercultural  College  Study,  students  at 
various  colleges  in  the  United  States  were  used  as  samples  of  the  popu¬ 
lation  of  college  students  in  the  United  States. 

c.  Formulation  of  ECHO  Questions 

The  data  resulting  from  an  ECHO  study  are  crucially  influenced  by 
the  form  of  the  ECHO  questions.  At  least  six  variables  may  be  discerned 
in  the  construction  of  ECHO  questions,  and  they  may  be  manipulated  in 
various  combinations.  The  variables  are: 

1.  Role  of  the  respondent  (for  example,  a  person  like  you, 
a  villager,  an  employee  at  this  company;  or  no  role 
assignment) . 

2.  Event  (for  example,  a  thing  to  DO,  a  thing  to  HAPPEN). 

3.  Valuation  of  the  event  (such  as,  good,  bad,  like,  not 
like,  ought,  ought  not). 

4.  Re  in  f  o  r  cemen  t  or  agency  (such  as,  approve,  disapprove, 
praise,  blame;  cause,  prevent). 
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5. 


Source  (reinforcer  or  agent:  for  example,  parents, 
myself;  a  good  job,  bad  luck). 

6.  Additional  context  (as,  in  the  village,  to  prevent  a 
riot,  during  a  strike,  while  on  vacation) . 

d.  Formulation  of  Instructions 

The  ^s  participating  in  an  ECHO  question  session  must  be  told  what 
to  do  with  the  materials  they  receive.  The  instructions  cover  both  the 
mechanics  of  using  the  materials  and  the  psychological  set  that  the  ^ 
should  assume  when  answering  the  questions.  Additional  context,  too 
detailed  for  inclusion  in  the  printed  questions  may  be  provided.  The 
instructions  also  help  motivate  the  ^s  to  cooperate  fully  by  giving  ^s 
a  plausibly  correct  reason  for  the  data  collection,  and  by  assuring  them 
either  of  anonymity  or  (if  that  is  not  possible)  of  confidential  treat¬ 
ment  of  the  responses. 

e.  Translation  of  Questions  and  Instructions 

When  data  are  to  be  collected  in  a  foreign  language,  the  questions 
and  instructions  must  be  translated.  This  task  is  more  exacting  than 
most  translation,  because  fine  shades  of  difference  may  be  crucial.  The 
E  must  decide  what  translation  procedures  to  use. 

f.  Number  of  Question  Cards 

The  E  has  some  latitude  in  deciding  how  many  question  cards  to 
present  to  each  All  of  the  studies  reported  here  used  10  cards  for 
each  question;  but  evidence  has  been  reported  (Appendix  II,  Manila  Study) 
that  7  cards  may  be  sufficient. 

g.  Selection  of  Supplementary  Measures 

It  is  often  advantageous  to  collect  biographical  data  from  the  ^s, 
or  to  record  other  objective  data  about  the  ^s .  These  measures  contribute 

See  Appendix  II,  Thai  Study  and  East  Los  Angeles  Skill  Center  Study; 
also  Phillips  (1959)  and  Bridge  and  Heller  (1968). 
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to  the  control  or  analysis  of  extraneous  factors  or  to  the  partitioning 
of  the  data  into  sub groupings.  Some  ECHO  studies  employ  supplementary 

psychological  measures  (for  example,  semantic  differential,  personality 
inventory  ),  which  can  be  used  to  evaluate  ECHO  construct  validity,  or 
to  improve  predictions. 

h.  Classification  of  Responses 

The  E  must  decide  how  many  classifications  of  the  data  will  be 
useful,  how  much  can  be  contributed  by  indigenous  classification,  whether 
to  use  staff  classification,  and  how  much  and  what  kind  of  interaction 
should  take  place  between  indigenous  and  staff  classifications  or  clas¬ 
sifiers.  When  indigenous  classifiers  are  used,  instructions  for  them 
must  be  written  and,  often,  translated. 

i.  ECHO  Message  Design 

The  ECHO  message  technique,  described  by  Barthol  and  Bridge  (1967, 
1968),  is  a  way  of  testing  the  concurrent  validity  of  ECHO  results. 

An  ECHO  message  is  usually  a  short  list  of  high-ranking  value  category 
titles  from  a  certain  group  of  respondents  which  is  presented  to  members 
of  the  group  to  be  judged  for  importance  or  acceptability  against  a 
similar  list  from  some  other  group  or  against  a  manufactured  list.  ECHO 
validity  is  confirmed  when  the  _Ss  choose  the  list  coming  from  their  own 
group  (the  indigenous  message)  over  the  alternative  message. 

If  ^  decides  to  hold  a  message  session,  he  must  decide  what  sort 
of  messages  to  use  (lists,  single  titles,  or  discursive  messages)  and 
what  alternative  messages  to  pair  with  the  indigenous  messages  for  judge¬ 
ment.  Instructions  for  the  message  session  must  also  be  written. 


T.  W.  Milburn,  personal  communication. 

'k'k 

See  Milburn,  Barthol,  and  de  Mille  (1968,  p.  41-42)  for  definitions 
of  concurrent  and  predictive  validity. 


19 


j.  Persuasive  Messages 

If  ^  wishes  to  estimate  predictive  validity,  he  can  plan  to  measure 
the  effects  of  persuasive  messages  based  on  ECHO  results.  (This  use  is 
distinct  from  the  use  of  persuasive  messages  to  achieve  practical  ends.) 

k.  Control  of  Extraneous  Factors 

In  the  selection  of  samples  and  during  question  sessions,  classi-- 
fication  sessions,  message  sessions,  and  use  of  persuasive  messages, 
extraneous  influences  may  introduce  bias  or  unreliability.  The  E  must 
take  precautions  so  that  _Ss  and  classifiers  are  not  distracted  or  system¬ 
atically  affected  by  adverse  or  unusual  conditions. 

l.  Methodological  Evaluation  and  Improvement 

The  ^  can  plan  to  estimate  reliability  of  responses  or  classifica¬ 
tions.  The  ^  can  introduce  modified  techniques  and  plan  to  evaluate 
their  outcomes. 

m.  Data  Organization  and  Reduction 

The  ^  has  various  options  in  the  analysis  of  the  data  and  presen¬ 
tation  of  the  results.  His  choices  will  be  dictated  largely  by  the 
inferences  he  wishes  to  make,  the  hypotheses  he  wishes  to  test,  or  the 
practical  uses  to  which  the  results  are  to  be  put.  The  plan  for  data 
organization  and  reduction  should  precede  data  collection. 

3.  Preparation  of  Materials 

Before  materials  are  prepared  in  quantity  for  a  question  session, 
all  content  that  has  not  been  tested  before  on  equivalent  ^s  should  be 
pretested  on  samples  equivalent  to  the  selected  ^s  or  on  samples  as 
nearly  like  the  S_s  as  possible. 


The  construction,  transmission,  and  evaluation  of  the  effects  of  per¬ 
suasive  messages  was  not  an  exercise  carried  out  during  the  ECHO  Pro¬ 
ject,  but  is  a  logical  further  step  both  in  the  exploration  of  ECHO 
validity  and  in  the  practical  use  of  ECHO  results. 
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When  the  content  has  proved  satisfactory  in  pre-tests,  question 
cards,  printed  instructions,  biographical  data  cards  or  forms,  and  supple¬ 
mentary  forms  are  prepared.  The  E  estimates  the  number  of  forms  needed 
and  prepares  printed  (or  mimeographed)  question  cards  with  punched  identi¬ 
fication  codes  that  assign  a  unique  number  to  each  ^  and  (within  one  ^’s 
card  set)  a  unique  number  to  each  card.  All  question  cards,  biographical 
cards,  and  supplementary  cards  for  each  ^  are  combined  in  the  correct 
order  in  an  envelope  or  other  packet;  orders  of  good  and  bad  (or  other 
question  alternatives)  are  counterbalanced  in  the  group;  supplementary 
and  biographical  cards  are  presented  to  the  S_  after  the  ECHO  question 
cards . 


Materials  for  message  sessions  usually  consist  of  booklets.  Each 
page  of  the  booklet  contains  a  pair  of  messages,  one  of  which  is  to  be 
selected  by  Different  combinations  are  counterbalanced  for  the  group 
of  ^s.  The  booklet  also  contains  instructions  as  needed. 

4.  Question  Session 

a.  Group  Administration 

ECHO  questions  are  usually  presented  to  _Ss  in  groups.  The  ^s 
follow  written  instructions  as  E  reads  the  same  instructions  aloud.  Time 
is  not  limited  (except  for  unusually  slow  ^s);  30  minutes  is  enough  time 
for  most  ^s. 

b.  Unobtrusive  Measures 

The  S^s  are  not  asked  to  supply  any  biographical  information  that 
can  as  well  be  unobtrusively  collected  by  ^  (for  example,  by  noting  on 
each  returned  envelope  any  observable  group  or  individual  characteristic 
in  which  E  is  interested,  such  as  work  shift  or  sex).  The  purpose  of 
this  constraint  is  to  minimize  the  S_s’  apprehensiveness  about  being  asked 
personal  questions. 
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c.  Individual  Interviews 

When  cannot  write,  the  data  can  be  collected  by  an  interviewer. 

d.  Self-Administered  Questions 

In  some  studies,  of  necessity,  the  question  materials  may  be  given 
to  each  to  be  completed  and  returned  at  ^’s  convenience. 

f.  Data  Collection  by  Mail 

Question  materials  can  be  mailed  to  _Ss  or  prospective  ^s ,  to  be 
returned  at  ^s *  option  and  convenience. 

These  four  techniques  do  not  produce  exactly  equivalent  data.  Each 
has  advantages  and  disadvantages.  In  general,  the  group  administration 
is  superior,  because  ^s  in  groups  comply  better  with  the  instructions 
than  _Ss  surveyed  at  their  convenience  or  by  mail,  and  because  the  group 
administration  avoids  the  unknown  effects  of  interactions  between  indi¬ 
vidual  ^s  and  an  interviewer.  Group  administration  is  also  the  least 
cos  tly . 

5 .  Indigenous  Classification 

The  responses  are  classified  by  one  or  more  teams  of  indigens,  who 
are  (usually  nonrespondent)  members  of  the  group  from  which  the  question- 
session  ^s  (respondents)  were  drawn.  After  each  classification,  the 
classification  codes  (category  numbers)  are  punched  into  the  data  cards. 
Under  field  conditions,  the  category  numbers  may  be  written  on  the  cards 
as  a  temporary  substitute  for  code  punching.  Codes  98  and  99  are  reserved 
for  miscellaneous  responses  and  non-responses  respectively;  code  99  is 
used  in  classifying  sources  where  responses  to  the  second  question 
(source  question)  are  omitted  by  Ss.  After  the  category  numbers  have 
been  punched  (or  written) ,  the  response  deck  is  ready  for  another 
classification. 
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6.  Classification  of  Sources 

Source  responses  are  usually  classified  by  staff  classifiers, 
according  to  a  standard  numbering  system.  When  the  data  come  from  an 
unfamiliar  culture,  however,  or  need  to  be  translated,  it  will  often  be 
helpful  to  have  all  source  response  classified  by  indigens  before  the 
standard  coding  system  is  applied. 

* 

7 .  Staff  Classification  of  Values 

Value  responses  are  classified  by  staff  members  or  trained 
field  assistants.  Staff  classification  has  two  main  purposes:  first, 
to  produce  accurately  constructed  categories  that  are  neither  overlapping 
nor  too  inclusive;  and  second,  to  combine  the  responses  of  different 
groups  or  subgroups  (such  as,  males  with  females,  rural  ^s  with  urban  _S s , 
Americans  with  Thai). 

It  is  usually  advantageous  to  allow  the  indigenous  and  staff  classi¬ 
fications  to  interact.  The  purpose  is  to  correct  the  misconceptions  of 
the  staff  classifiers  by  exposing  them  to  the  special  knowledge  of  the 
indigenous  classifiers.  The  interaction  may  be  accomplished  in  three 
ways : 

a.  Concurrent  interaction,  in  which  the  staff  team  works 
with  one  or  more  indigens  as  helpers  and  informants. 

b.  Sequential  interaction,  in  which  the  staff  team  studies 
the  results  of  the  indigenous  classification  before  and 
during  the  staff  classification.  The  staff  team  can 
read  the  indigenous  classification  codes  on  the  response 
cards  and  can  review  the  indigenous  category  titles. 


In  the  Pentalith  report  (de  Mille  and  Barthol,  1969),  staff  classifi¬ 
cation  was  called  professional  classification  .  Staff  classification, 
used  here,  is  a  better  term,  because  staff  classifiers,  though  trained 
to  do  classification,  do  not  have  to  be  at  the  level  of  competence 
or  responsibility  usually  connoted  by  the  word  professional. 
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c.  Terminal  interaction,  in  which  the  results  of  the  two 

classifications  are  compared  by  computer  program  ROCKEM 
(see  Appendix  III),  Errors  in  the  staff  classification 
can  be  discovered  by  this  technique  and  corrected. 

8.  Separation  into  Subgroups 

After  the  staff  classification  has  been  completed  and  the  resulting 
codes  have  been  punched  into  the  data  cards,  the  respondent  samples  that 
were  combined  for  the  classification  are  separated  by  sorting  the  cards 
back  into  their  original  groupings  or  into  new  subgroupings  (such  as, 
male  and  female,  younger  and  older). 

The  data  cards  for  each  group  or  subgroup  are  processed  by  computer 
program  UNIKOUNT  (see  Appendix  III)  ,  which  counts  the  responses  in  each 
category  and  the  ^s  represented  (by  at  least  one  response)  in  each  cate¬ 
gory.  The  output  consists  of  tables  showing  the  frequencies,  percentages, 
and  rank  numbers  associated  with  each  category  title. 

9 .  Comparisons  of  Groups  and  Subgroups 

The  UNIKOUNT  tables  are  used  to  prepare  sets  of  data  cards  for 
program  PERZPROB  (see  Appendix  III) ,  each  set  containing  the  category 
code  numbers  and  associated  titles  and  percentages  from  one  UNIKOUNT 
table.  Program  PERZPROB  can  compare  up  to  15  such  data  sets,  printing 
out  a  table  in  which  the  category  titles  are  assigned  to  rows,  and  the 
groups  or  subgroups  are  assigned  to  columns;  the  body  of  the  table  shows 
the  percentage  of  each  group  of  ^s  represented  in  each  category.  The 
program  is  used  for  value  comparisons,  source  comparisons,  and  biograph¬ 
ical  and  supplementary  comparisons. 

Program  ROCKEM  can  be  used  to  determine  which  sources  are  associated 
with  which  value  categories  in  the  various  groups.  The  program  prints  a 
table  for  each  value  category  showing  the  number  of  responses  in  which 
each  source  was  associated  with  that  value  category;  or  the  arrangement 
can  be  reversed,  showing  how  each  value  category  was  associated  with  a 
given  source. 
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10 .  Statistical  Analysis 

Five  kinds  of  statistical  analysis  of  the  ECHO  data  can  be  added 
to  the  procedures  described  in  steps  8  and  9.  The  first  three  are  already 
operationalized  in  computer  programs;  the  last  two  are  proposed  as  desir¬ 
able  kinds  of  data  reduction,  to  be  developed  later. 

a.  Statistical  Tests  of  Percentage  Differences 

Program  PERZPROB  displays  the  percentage  of  each  group  of  ^s  repre¬ 
sented  in  each  ECHO  category.  In  addition,  the  program  computes  the  exact 
probability  that  a  difference  at  least  as  large  as  that  observed  between 
two  groups  (in  one  category)  would  arise  by  chance.  The  ^  uses  the 
PERZPROB  probability  table  to  evaluate  the  significance  of  pairs  of  per¬ 
centages  in  the  percentage  table  (see  Appendix  III) . 

b.  Correlation  of  Groups  or  Subgroups 

Pearson  product-moment  correlations  (Guilford,  1956)  can  be  computed 
between  two  groups  across  the  frequencies  of  ^s  represented  in  ECHO  cate¬ 
gories  (see  Appendix  II,  Intercultural  College  Study).  These  correlations 
show  the  overall  similarity  or  dissimilarity  of  ECHO  responses  in  the 
two  groups;  they  do  not  show  how  any  specific  ECHO  categories  contribute 
to  the  similarity  or  dissimilarity.  (Information  about  intergroup  dif¬ 
ferences  in  specific  ECHO  categories  is  found  in  the  PERZPROB  tables.) 

Any  standard  computer  correlation  program  can  be  used. 

c.  Factor  Analysis  of  ECHO  Data 

The  intergroup  correlations  can  be  factor  analyzed  when  a  large 
number  of  categories  has  been  used  to  compute  the  correlations  and  when 
the  groups  are  chosen  so  as  to  satisfy  the  assumptions  of  factor  analysis. 
In  the  Intercultural  College  Study  (Appendix  II) ,  a  standard  factor  anal¬ 
ysis  computer  program  isolated  United  States,  Thai,  Kenya,  and  Manila 
factors  as  predicted.  (Factors  based  on  intercorrelations  of  categories 

For  constraints  on  the  interpretation  of  such  factors,  see  Guilford 
(1952). 
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or  on  intercorrelations  of  _Ss  would  be  more  useful,  but  the  problems  of 
computing  these  correlations  from  ECHO  data  have  not  been  worked  out.) 

d.  Latent  Category  Analysis 

When  the  same  responses  are  classified  several  times  by  different 
teams  of  classifiers,  inspection  shows  that  some  categories  have  a  core 
of  responses  that  is  invariant  from  classification  to  classification. 

The  invariant  core  represents  the  latent  category,  which  is  less  accu¬ 
rately  represented  by  the  corresponding  categories  from  the  different 
classifications.  Analysis  of  ECHO  data  for  latent  categories  might  be 
accomplished  with  latent  partition  analysis  (Wiley,  1967)  and  might 
delineate  the  important  values  in  a  culture  with  less  error  than  present 
classification  procedures. 

e.  Subject  Grouping 

It  should  be  possible  to  test  the  assumption  that  a  group  of  ECHO 
^s  is  homogeneous  by  analyzing  the  pattern  of  response  concordances  with¬ 
in  each  pair  of  ^s .  This  analysis  is  like  correlating  all  pairs  of  _S s . 
Exact  procedures  have  not  yet  been  tested. 

11 .  Display  of  Results 

Computer  programs  UNIKOUNT,  ROCKEM,  and  PERZPROB  provide  several 
kinds  of  printed  displays  that  may  be  used  in  interpreting  ECHO  results. 
Among  these  is  the  complete  list  of  classified  responses  that  can  be 
obtained  from  program  ROCKEM  when  keypunching  of  responses  has  been 
limited  to  one  card  per  response.  When  a  display  deck  has  been  prepared, 
showing  all  responses  in  their  original  length,  lists  of  classified 
responses  can  be  obtained  by  sorting  and  listing  the  display  deck. 

12 .  Identification  of  Significant  Items 

Significant  items  of  difference  between  groups  can  be  identified 
by  a  clerical  assistant  using  the  PERZPROB  probability  table.  Every 
number  in  the  table  that  is  equal  to  or  less  than  a  selected  significance 
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level  (for  example,  .05,  .01)  can  be  circled;  the  pairs  of  percentages 
corresponding  to  the  circled  numbers  can  then  be  abstracted  and  a  more 
convenient  table  made  up  for  interpretation  by 

A  list  of  items  in  which  two  groups  are  (significantly)  different 
and  another  list  of  items  in  which  they  are  (correspondingly)  similar 
can  be  made  up  (see  Appendix  II,  Tables  11,  12,  13).  In  such  contrasting 
lists,  the  patterns  of  intergroup  similarity  and  difference  can  easily 
be  seen. 

13.  Rational  Analysis  and  Hypothesis  Formation 

The  E*s  training  and  experience  are  brought  to  bear  on  the  results 
to  formulate  compelling  and  defensible  findings,  supported  by  the 
results,  and  testable  hypotheses,  suggested  by  the  findings.  Steps  4 
to  12  can  be  carried  out  in  a  routine  fashion,  but  step  13,  like  steps  1 
through  3,  requires  professional  judgement. 

14.  ECHO  Products 

Five  kinds  of  information  result  from  ECHO  studies: 

a.  Value  Hierarchies 

A  value  hierarchy  describes  the  values  of  a  group  that  are  suffi¬ 
ciently  salient  in  the  awareness  of  members  of  the  group  to  be  explicitly 
and  spontaneously  stated,  and  gives  their  order  of  importance  for  the 
group . 


b.  Source  Hierarchies 

A  source  hierarchy  describes  the  sources  of  influence  perceived 
and  expressed  by  members  of  the  group  and  gives  their  order  of  importance. 

c.  Group  Comparisons 

Group  comparisons  list  and  interpret  the  differences  and  similarities 
between  groups,  as  the  group  members  perceive  the  values  and  sources  of 
influence  in  their  lives. 
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d.  Theory 

ECHO  hypothesis  formation  and  testing  add  to  the  psychological 
theory  of  value  and  to  related  theoretical  areas,  such  as  social  psychol¬ 
ogy  and  personality  theory. 

e.  Method 

Methodological  findings  help  to  improve  the  ECHO  method  and  may 
influence  related  methods. 

15 .  Uses  of  ECHO  Products 

ECHO  products  may  be  used  in  many  ways,  prominent  among  which  are 
the  following  five: 

a.  To  Understand 

The  foremost  use  of  ECHO  results  is  to  enhance  understanding  of 
the  value-and- influence  patterns  in  groups  of  people,  whether  familiar, 
unfamiliar,  or  esoteric. 

b.  To  Predict 

Information  about  values  can  be  used  to  improve  prediction  about 
behavior  in  the  populations  surveyed  (see  Milburn,  Barthol,  and  de  Mille, 
1968,  pp .  14-17).  Such  predictions  can  serve  theoretical  or  practical 
purposes . 

c.  To  Train 

ECHO  information  should  prove  useful  in  training  for  culture  con¬ 
tact  and  in  the  proximate  evaluation  of  such  training. 


This  brief  list  of  uses  is  subsumed  by  the  list  of  recommendations  for 
applying  ECHO,  Sec.  VI.  Uses  b,  c,  and  e  are  plausible  uses  of  ECHO 
information  but  have  not  been  tested;  uses  a  and  d  have  been  tested 
and  provisionally  validated. 
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d.  To  Communicate 

The  value  categories  and  the  detailed  language  in  which  responses 
are  given  can  be  material  for  constructing  messages  that  are  acceptable 
and  understandable  to  members  of  the  populations  surveyed. 

e.  To  Persuade 

The  persuasive  power  of  messages  can  be  enhanced  by  congruity  with 
the  value  system  and  favored  expressions  of  target  groups. 

Items  16,  17,  20,  and  21,  that  follow,  are  optional  ECHO  procedures; 
items  18  and  19  are  possible  ECHO  procedures,  and  are  proposed  for  future 
trial. 


16 .  Other  Data 

a.  Data  from  Other  Groups 

The  value  system  of  a  particular  group  can  be  put  into  an  inter¬ 
group  or  cross-cultural  context  by  the  addition  of  responses  from  a 
different  group  or  set  of  groups;  all  of  the  data  are  then  classified 
together.  (See  Appendix  II,  Intercultural  College  Study.) 

b.  Manufactured  Data 

Manufactured  responses  can  be  included  in  a  classification  to  test 
hypotheses.  (See  Appendix  II,  High  School  Training  Group  Study.) 

17.  Translation  and  Preparation  of  Reclassification  Deck 

a.  Translation 

Data  from  different  groups  can  be  translated  into  a  common  language 
for  unified  classification. 

b.  Reclassification  Deck 

Computer  program  PRINDEK  (see  Appendix  III)  prints  cards  containing 
complete  responses  (one  response  per  card)  for  unbiased  classification  of 
mixed  samples  or  classification  of  translated  samples. 
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18. 


Conditional  Responses 

Indigens  review  the  category  titles  and  give  conditions  under  which 
the  stated  values  would  not  apply  or  could  properly  be  violated.  These 
conditional  responses  could  be  used  to  test  hypotheses  about  the  unstated 

•k 

background  conditions  of  expressed  values. 

kk 

19 .  Special  Classification 

ECHO  classification  could  be  used  as  a  primary  method  of  investi¬ 
gation  rather  than  a  step  in  the  analysis  of  responses.  Several  distinct 
kinds  of  problems  may  be  listed. 

a.  Classification  Parameters 

The  nature  of  the  classification  process  could  be  studied  by  system¬ 
atically  varying  the  instructions  to  classifiers  (for  example,  classifiers 
could  be  told  how  many  categories  to  make,  or  given  logical  rules  for 
grouping  cards  together) .  Sequential  effects  of  sorting  and  grouping 
could  be  studied. 

b.  Characteristics  of  Classifiers 

Classifiers  from  different  groups  could  classify  the  same  data  to 

kkk 

reveal  differences  in  classification  behavior  attributable  to  groups. 
Classifiers  with  different  psychological  traits  could  classify  the  same 
data  to  reveal  relationships  between  classification  traits  and  other 
traits . 

c.  Theoretical  Constructs 

Classifiers  could  be  instructed  to  group  cards  together  in  theoret¬ 
ically  based  categories.  Instead  of  putting  responses  together  that 


The  use  of  conditional  responses  was  proposed  by  Bridge  (Bridge  and 
Heller,  1968) . 

See  also  Milburn,  Barthol,  and  de  Mille  (1968,  pp.  21-22). 

An  approach  to  this  technique  was  used  in  early  ECHO  studies,  where 
females  classified  male  data  and  vice  versa  (Barthol  and  Bridge,  1967). 
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"say  or  mean  the  same  thing,"  classifiers  could  be  given  a  more  specific 
rule,  such  as  "group  responses  together  that  show  love  for  others." 

d.  Cross-Cultural  Sorting 

Classifiers  could  be  asked  to  sort  responses  from  one  culture  into 
categories  from  another  culture,  to  reveal  the  congruities  and  incongru¬ 
ities  between  the  two  systems. 

20.  Messages  and  Other  Concurrent  Checks 

a.  Message  Session 

The  message  session  was  one  of  the  early  ECHO  techniques,  developed 
to  assess  concurrent  validity  of  ECHO  findings.  Value  statements  derived 
from  one  population  (indigenous  statements)  are  presented  to  members  of 
the  population  along  with  competing  statements  from  other  populations 
(Barthol  and  Bridge,  1967)  or  manufactured  statements  (Barthol  and  Bridge, 
1968).  When  a  significant  majority  of  the  group  select  the  indigenous 
statements,  _E  concludes  that  the  statements  represent  important  indigenous 
values . 

b.  Other  Concurrent  Measures 

Any  other  measures  of  values  or  of  behavior  predicted  from  values 
can  be  used  to  assess  the  concurrent  validity  of  ECHO  findings. 

21.  ECHO  Information  and  Hypothesis  Testing 

When  alternative  sources  of  information  about  the  subject  popula¬ 
tion  are  available,  ECHO  findings  can  be  tested  against  them  and  verified, 
refuted,  or  qualified.  When  formal  hypotheses  have  been  stated,  they 
can  be  accepted  or  rejected.  In  addition,  ECHO-generated  hypotheses 
could  be  tested  in  laboratory  experiments  (for  example,  small  group 
studies,  game  studies). 

* 

This  technique  was  tried  in  Project  ECHO  when  the  responses  of  the  East 
Los  Angeles  Skill  Center  trainees  (see  Appendix  II)  were  sorted  into 
the  Intercultural  College  value  categories;  the  pattern  of  congruity- 
incongruity  was  marked  and  readily  interpretable. 
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B.  ECHO  CLASSIFICATION,  DISCUSSION  AND  DETAILED  DESCRIPTION 

When  50  or  100  ECHO  respondents  each  fill  out  7  to  10  question 
cards,  they  generate  a  vast  amount  of  heterogeneous  information  that 
would  be  very  difficult  to  interpret  if  not  organized.  One  purpose  of 
classification  is  to  introduce  a  relatively  simple  and  interpretable 
order  into  this  complexity.  A  second  purpose  is  to  arrive  at  an  index 
of  importance  for  each  category  of  value.  Categories  in  which  many  of 
the  respondents  are  represented  are  considered  more  important  than  cate¬ 
gories  in  which  only  a  few  respondents  are  represented;  usually  a  dis¬ 
tinct  hierarchy  of  importance  emerges. 

1.  Indigenous  and  Staff  Classification 

In  the  ECHO  method,  two  kinds  of  classification  are  routinely 
used:  indigenous  classification,  done  by  a  team  selected  from  the  same 

group  that  the  respondents  came  from;  and  staff  classification,  done 
either  by  ECHO  staff  members  or  by  trained  field  assistants.  Each  makes 
its  own  unique  contribution. 

The  staff  classifiers  tend  to  group  the  cards  into  more  precise 
and  better  differentiated  categories  than  do  the  indigenous  classifiers. 
And  they  can  work  longer  without  fatigue  at  their  exacting  task  than  the 
indigenous  team.  In  the  Pentalith  Study  (de  Mille  and  Barthol,  1969) 
the  staff  team  classified  as  many  as  869  cards  in  one  session,  whereas 
the  largest  number  of  cards  classified  by  any  of  the  four  indigenous 
teams  in  one  session  was  310.  The  ability  to  classify  a  larger  number 
of  cards  makes  it  easier  to  include  cards  from  different  groups  in  the 
same  classification,  so  that  intergroup  comparisons  can  be  made  in  the 
same  frame  of  reference. 

On  the  other  hand,  and  despite  the  merits  of  staff  classification, 
indigenous  classification  is  often  indispensable.  An  advantage  of  the 
ECHO  method  is  that  ideas  and  values  of  the  group  being  studied  emerge 
with  relatively  little  distortion;  indigenous  classification  assures 
this . 
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The  quality  of  indigenous  classifications,  however,  may  vary  con¬ 
siderably.  The  resultant  categories  may  be  too  inclusive  or  too  much 
alike;  cards  may  be  misclassif ied,  titles  may  be  inaccurate.  It  is  help¬ 
ful  to  have  two  or  more  indigenous  classifications  to  compensate  for 
such  deficiencies. 

Although  indigenous  classifiers  usually  know  little  about  classify¬ 
ing  as  a  procedure,  they  know  a  lot  about  the  values  and  language  of 
their  own  group.  If  the  staff  team  misclassif ies  a  certain  response 
because  of  being  unfamiliar  with  indigenous  values  or  language,  the  indig¬ 
enous  classifiers  usually  supply  corrective  information.  For  example 
in  the  Pentalith  study,  one  respondent's  card  said  that  a  good  thing 
would  be  "to  find  sufficient  time  to  complete  my  education,"  a  response 
placed  by  the  staff  classifiers  with  others  they  labeled.  Get  More  Educa¬ 
tion  and  Improve  Self.  The  indigenous  team.,  however,  put  the  same  card 
with  a  group  labeled  Have  Good  Working  Schedules.  The  implication  missed 
by  the  staff  team  was  that  working  schedules  can  determine  whether  or 
not  attending  school  will  be  possible  during  time  off. 

The  three  ways  in  which  indigenous  and  staff  classifications  may 
interact  to  reduce  error  have  been  described  on  pages  23  and  24. 

2 .  Six  Steps  in  Indigenous  Classification 

Three  indigenous  classifiers  work  under  the  supervision  of  an  ECHO 
staff  member  or  field  assistant.  The  cards  to  be  classified  are  shuffled 
and  divided  among  the  three  classifiers.  (A  single  classification  session 
is  confined  to  one  set  of  ECHO  questions,  that  is,  good  or  bad.) 

Step  One.  Each  classifier  works  by  himself,  reading  the  good 
thing  (or,  in  another  session,  the  bad  thing)  written  on  each  card  and 
grouping  cards  together  that,  in  his  judgment,  say  or  mean  the  same 
thing. 
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Step  Two.  The  three  classifiers  work  together,  combining  all  their 
separate  sets  of  categories.  One  member  of  the  team  reads  aloud  the 
responses  from  one  of  his  categories  while  the  other  two  add  cards  from 
their  own  groups  that  fit  that  category.  The  result  is  one  unified 
classification  system. 

Step  Three.  The  team  reviews  the  categories  by  reading  the  cards 
again,  correcting  errors  of  classification,  combining  equivalent  cate¬ 
gories  or  splitting  any  that  seem  too  complicated,  and  assigning  hard- 
to-classify  cards  either  to  existing  categories  or  to  a  miscellaneous 
pile. 


Step  Four.  The  team  writes  a  label  for  each  pile,  following 
instructions  that  the  label  should  be  in  the  form  of  an  answer  to  the 
ECHO  question  and  should  summarize  all  responses  in  the  category.  (The 
administrator  is  careful  not  to  prescribe  either  the  groupings  of  cards 
or  the  wording  of  titles,  giving  neither  examples  nor  hints.) 

Step  Five.  The  administrator  asks  the  team  whether  any  of  the 
categories  contain  unsuitable  answers.  Occasionally,  classifiers  reject 
a  category,  saying,  for  example,  "These  so-called  good  things  are  really 
bad  things  and  should  not  have  been  included  in  this  classification." 

Step  Six.  In  response  to  the  instruction,  "Now  put  these  piles 
in  their  order  of  importance  according  to  how  you  think  or  feel  about 
them,"  the  team  ranks  its  categories  in  order  of  "importance."  (Rejected 
categories  go  to  the  low  end  of  the  hierarchy.)  When  the  categories  are 
ranked,  their  label  cards  are  numbered  in  that  order,  number  1  being  the 
most  important  category  and  all  the  numbers  being  referred  to  as  cate¬ 
gory  numbers.  These  numbers  remain  with  their  labels  throughout  the 
study.  (Rejected  categories  are  designated  by  a  decimal  jump  in  num¬ 
bering.  ) 
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C,  ECHO  SOURCES  OF  APPROVAL  AND  DISAPPROVAL 

Most  of  the  source  responses  collected  in  Project  ECHO  were 
answers  to  the  questions:  "Who  would  approve?"  and  "Who  would  disapprove?" 
Table  1  shows  a  list  of  ECHO  sources  of  approval  and  disapproval,  with 
their  standard  code  numbers.  Each  of  these  standard  sources  is  a  cate¬ 
gory  title,  subsuming  various  source  responses.  Source  number  1,  for 
example,  subsumes  responses  such  as:  myself,  I,  me,  you,  my  conscience. 

Standardization  of  sources  has  been  possible  because  the  same  source 
questions  were  used  in  numerous  studies  and  because  source  responses  are 
more  concrete  and  more  common  to  various  groups  than  value  responses. 

Various  findings  about  ECHO  sources  have  been  reported  in  the  Thai 
Study  and  Oakland  High  School  Study  (Appendix  II),  the  Pentalith  Study 
(de  Mille  and  Barthol,  1969) ,  and  by  Barthol  and  Bridge  (1967) .  Most  of 
these  findings  describe  the  relative  importance  of  selected  sources  in 
different  groups;  some  findings  describe  the  interactions  between  sources 
and  values  in  different  groups. 

Bavelas  (1942)  stressed  the  importance  of  discovering  the  specific 
connections  between  the  approved  or  disapproved  behaviors  and  the  sources 
of  approval  or  disapproval.  Methodological  findings  in  Project  ECHO 
confirm  those  of  Kalhorn  (1944) ,  who  demonstrated  that  this  kind  of 
analysis  was  both  feasible  and  fruitful. 


Barthol  and  Bridge  (1967)  referred  to  the  analysis  of  source-value  inter¬ 
actions  as  an  analysis  of  the  "power  structure,"  a  term  that  may  mislead 
by  suggesting  that  actual  social  interactions  have  been  observed.  The 
source-value  interactions  implied  in  the  verbal  responses  of  ECHO  ^s  are 
those  perceived  or  assumed  by  the  ^s ;  their  relation  to  actual  social 
power,  past  or  present,  is  undetermined.  The  term  "power  structure" 
later  gave  way  to  "value  and  influence  patterns"  (Barthol  and  Bridge, 
1968). 
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TABLE  1 


STANDARD  SOURCES  OF  APPROVAL  AND  DISAPPROVAL 


1. 

Myself 

40. 

Teacher 

2. 

Spouse 

41. 

Principal 

3. 

Parents 

42. 

Faculty 

4. 

Mother 

43. 

Administration 

5. 

Father 

44. 

Boss 

6. 

Family 

45. 

Organizations  (misc. ) 

7. 

Brothers  or  sisters 

46. 

School 

8. 

Relatives 

47. 

Employee 

9. 

Children 

48. 

Employer 

49. 

Superiors 

o 

CM 

Friends 

21. 

Peers 

50. 

United  States 

22. 

Students 

51. 

Government 

23. 

Associates 

52. 

Church 

24. 

Boy  or  girl  friend 

53. 

Minister  or  priest 

54. 

God 

26. 

Fiance 

55. 

Professionals 

27. 

Neighbors 

56. 

Soldiers 

28. 

Roommate 

57. 

Police 

29. 

Fellow  countrymen 

58. 

Civil  servants 

30. 

Others,  people 

59. 

Liberals 

31. 

Everyone 

60. 

Those  who  approve /disapprove 

32. 

Society 

61. 

Good  people 

33. 

Person  involved 

62. 

President 

34. 

Poor  people 

35. 

Own  group 

96. 

Don’t  know 

36. 

People  opposed 

37. 

Minority  groups 

97. 

Nobody 

38. 

Other  nationals 

98. 

Miscellany 

39. 

Old  people 

99. 

Blank,  nothing  written 
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D.  PARAMETERS  AND  REQUIREMENTS  OF  THE  ECHO  METHOD 

Section  A  briefly  described  and  discussed  twenty-one  procedures 
(and  products)  of  the  ECHO  method;  Sec.  B  gave  a  detailed  description  and 
some  discussion  of  classification  procedures;  and  Sec.  C  presented  the 
standard  ECHO  sources  of  approval  and  disapproval  and  discussed  their 
significance  in  the  ECHO  method.  This  section  will  give  further  details 
about  the  use  of  the  ECHO  method,  referring  for  some  points  to  supporting 
evidence. 


1.  Variables  in  the  ECHO  Questions 

Six  variables  in  the  ECHO  questions  have  been  briefly  defined 
(p.  17).  These  variables  will  be  discussed  further. 


Role  of  the  Respondent 

Bavelas  (1942)  pointed  out  that  the  role  of  the  respondent  could 
be  varied  and,  particularly,  that  it  could  be  more  or  less  personal  (that 
is,  more  or  less  identified  with  the  respondent  himself).  Bavelas  com¬ 
pared  the  indirect,  impersonal  question  with  a  clinical  projective  play 
situation,  where  a  child  is  asked  to  play  with  family  dolls  and,  without 
being  specifically  instructed  to,  plays  a  game  about  his  own  family. 

Role  specifications  in  ECHO  questions  could  be,  for  example: 


Though  Barthol  and  Bridge  (1967)  referred  to  ECHO  as  a  "projective" 
survey,  it  should  be  pointed  out  that,  while  ECHO  fits  some  criteria  for 
projective  devices,  it  does  not  fit  others  (Llndzey,  1961).  Like  pro¬ 
jective  devices,  ECHO  does  elicit  multiple  and  varied  responses  that 
could  be  multidimensionally  analyzed;  and  there  is  no  requirement  that 
ECHO  responses  be  correct.  On  the  other  hand,  and  in  contrast  to  pro¬ 
jective  devices,  the  ECHO  stimulus  is  not  ambiguous  (most  ^s  understand 
it  the  same  way);  nor  does  it  usually  tend  to  evoke  fantasy  responses; 
nor  are  ECHO  responses  usually  employed  to  give  a  holistic  picture  of  an 
individual  personality.  Two  indispensable  criteria  for  projective  devices 
are:  sensitivity  to  unconscious  or  latent  aspects  of  personality,  and 

lack  of  ^  awareness  of  the  variables  in  which  ^  is  interested.  ECHO 
responses  reflect  group  values  of  which  the  ^  is  usually  well  aware,  as 
he  may  be  well  aware  of  the  kind  of  variable  (that  is,  values)  in  which  ^ 
is  interested.  Bavelas  limited  his  comparison  with  projective  devices  to 
the  ability  of  both  techniques  to  remove  any  psychological  threat  that 
might  inhibit  ^'s  responsiveness. 
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You 


A  person  like  you 
A  person  like  your  friend 
A  person  like  your  neighbor 
A  member  of  your  class 
A  person  older  than  you 
A  person  living  in  another  country 
A  person  you  would  not  like 
A  person  very  different  from  you 

These  different  specifications  seem  progressively  less  identified  with 
the  _S.  Bavelas  assumed  that  the  role  "a  person  like  you"  would  be 
sufficiently  impersonal  to  allow  free  expression  of  ideas  and  also  to 
elicit  values  prevalent  in  the  respondent  group  rather  than  values 
peculiar  to  the  individual  An  alternative  to  such  specifications  is 
the  completely  unspecified  role,  as  in  the  question,  "What  is  a  good 
thing  to  do?" 

The  choice  of  role  specification  depends  on  whether  ^  wishes  to 
structure  the  responses  within  some  particular  role  (for  example,  to  test 
a  hypothesis  or  measure  empathy)  or  wishes  to  elicit  the  most  general 
and  spontaneous  array  of  value  responses  from  the  group.  Most  ECHO 
research  has  used  "a  person  like  you"  or  the  unspecified  role. 

Event 

Most  ECHO  research  has  used  the  behavioral  or  thing-to-do  event; 
some  studies  have  used  the  non-behavioral  or  thing- to-happen  event.  In 
general,  the  event  puts  the  ^  in  a  more  active  role  than  the  happen 
event,  though  ^s  may  reinterpret  the  question  to  satisfy  their  own  needs 
to  be  either  active  or  passive.  Other  events  are  possible,  such  as 

For  example,  an  active  may  say,  "A  good  thing  to  happen  is  for  me  to 
go  out  and  find  a  job,"  while  a  passive  ^  may  say,  "A  bad  thing  to  do  is 
to  be  executed  in  the  gas  chamber." 
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things  to  think  about  or  things  to  buy,  own,  wear,  or  eat.  The  more 
specific  the  event,  the  greater  the  risk  of  reducing  the  number  of 
responses  from  each 

Valuation  of  the  Event 

The  questions  "What  is  a  good/bad  thing  to  do?"  elicit  personal 
preferences  and  aversions  as  well  as  social  obligations  and  prohibitions; 
the  Ss  mention  things  they  would  or  would  not  like  to  do  as  well  as  things 
they  ought  or  ought  not  to  do.  To  a  considerable  extent  the  personal  and 
the  social  responses  are  in  agreement  (see  Milburn,  Barthol,  and  de  Mille, 
1968,  pp.  14-17;  also.  Appendix  II,  Thai  Study). 

Kalhorn  (1944)  suggested  that  like  and  dislike  could  be  substituted 
for  good  and  bad.  In  the  Thai  Study  (Appendix  II) ,  the  like/dislike- 
ought/ought  not  (LODS)  question  set  was  developed  to  solve  a  translation 
problem  and  was  retained  as  a  valuation  variant  that  could  discriminate 
preferences  and  aversions  from  obligations  and  prohibitions. 

Kalhorn  also  suggested  fear  as  a  valuation  (for  example,  "Something 
you  would  be  afraid  to  do"),  but  such  further  variants  were  not  tested 
in  Project  ECHO. 

Reinforcement  or  Agency 

These  two  variables  are  treated  together  because  they  have  not  been 
used  in  the  same  questions  and  are  considered  to  be  alternatives  in 
question  formulation.  Reinforcement  is  a  consequence  of  behavior  per¬ 
ceived  by  ^  and  is  associated  with  questions;  agency  is  a  relationship 
of  cause  and  effect  perceived  by  ^  and  is  associated  with  happen  questions. 
Other  arrangements  would  be  possible  (for  example,  "What  is  a  bad  thing 
to  do,  and  what  would  cause  someone  to  do  it?"),  but  such  arrangements 
have  not  been  tested. 

See  Glossary,  Appendix  I. 
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During  Project  ECHO  it  was  noted  that  some  ^s  who  answer  questions 
involving  approval  and  disapproval  may  persist  in  giving  approval 
responses  to  disapproval  questions.  In  the  East  Los  Angeles  Skill  Center 
(ELASC)  Study  (Appendix  II),  where  reading  skills  were  relatively  low 
even  among  the  ’’readers,"  there  were  numerous  responses  of  the  following 
kind: 

What  is  a  bad  thing  to  do? 

To  rob . 

Who  would  disapprove? 

The  robber. 

It  has  been  necessary  for  E  to  take  special  pains  to  make  sure  that 
^s  realize  that  bad  questions  call  for  disapproval  responses;  this  is 
achieved  by  appropriate  emphasis  in  the  instructions. 

A  similar  problem  has  arisen  in  preparing  the  LCDS  question  set. 
When  the  questions  are,  "What  is  something  you  would  not  like  to  do?" 

"Who  would  disapprove?"  the  ^  is  not  sure  whether  the  second  question 
means  disapproval  of  doing  the  thing  or  disapproval  of  not  doing  or  not 
liking  to  do  the  thing.  The  ambiguity  can  be  removed  by  rephrasing  the 
question  thus:  "Who  would  disapprove  if  you  did  it?" 

These  problems  point  up  the  necessity  for  pretesting  new  sets  of 
questions,  or  old  sets  of  questions  that  are  to  be  used  in  a  new 
population . 

Source 

The  standard  list  of  approvers  and  disapprovers  (Table  1)  was 
developed  largely  from  student  samples.  Though  the  list  has  considerable 
generality,  other  populations  generate  additional  sources;  in  the  East 
Los  Angeles  Skill  Center  Study,  for  example,  several  ^s  gave  the 
unemployed  as  a  source.  Source  lists  developed  from  other  kinds  of 
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questions  may  be  quite  different.  In  the  Pentalith  Study  (de  Mille  and 
Barthol,  1969),  where  a  happen  question  specified  "working  at  Pentalith," 
the  source  list  included  many  items,  such  as  working  conditions,  auto¬ 
mation,  accident,  scheduling,  faulty  equipment,  and  union,  that  do  not 
appear  in  the  list  of  approvers  and  disapprovers . 

Additional  Context 

The  context  that  is  added  to  the  five  variables  discussed  in  the 
previous  paragraphs  can  make  the  role  or  the  situation  more  specific. 

In  the  question,  "What  is  a  good  thing  for  a  person  like  you  to  do  in 
the  village?"  the  additional  context  "in  the  village"  could  be  a  further 
specification  of  the  role  ("a  person  like  you")  if  the  respondent  were 
himself  a  villager;  or  it  could  be  a  specification  of  the  situation  if 
the  respondent  were  someone  whose  role  had  never  been  defined  by  village 
relationships . 

Another  kind  of  additional  context  is  the  specification  of  past  or 
future  time  or  conditional  mood,  to  modify  the  event.  Such  specifications 
may  include : 

A  thing  someone  did 
A  thing  someone  will  do 
A  thing  someone  could  do 
A  thing  that  has  happened 
A  thing  that  will  happen 
A  thing  that  could  happen 

Past-tense  happen  questions  tend  to  elicit  satisfactions  (about 
good  things)  or  disappointments  (about  bad  things) .  Conditional  happen 
questions  (a  thing  that  could  happen)  tend  to  elicit  hopes  (for  good 
things)  or  fears  (of  bad  things).  In  the  Pentalith  Study  (de  Mille  and 
Barthol,  1969),  past  and  conditional  happen  questions  were  used  to  study 
hopes  and  satisfactions,  fears  and  disappointments  of  employees. 

Additional  context  tends  to  limit  the  scope  of  the  question  and  may 
reduce  the  variety  of  the  responses,  and  in  turn,  the  number  of  categories. 
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Complexity  and  Conflict 

Bavelas  (1942)  experimented  with  more  complex  questions,  in  which 
a  conflict  of  values  was  introduced.  He  asked  young  girls  the  question: 
"What  is  a  good  thing  to  do  and  someone  [source]  will  praise  [reinforce¬ 
ment]  you,  but  your  girl  friends  in  the  club  will  scold  you?"  In  this 
question  the  clause,  "but  your  girl  friends  in  the  club  will  scold  you," 
is  a  fixed  additional  context  that  introduces  a  social  conflict  over 
values.  Some  of  the  responses  were  similar  to  the  conditional  responses 
described  on  page  30,  since  they  reflected  conditions  under  which 
certain  values  would  not  apply  or  could  be  violated. 

Conflict  could  also  be  represented  in  ECHO  questions  by  repetition 
of  question  elements  rather  than  by  additional  context,  thus:  "What  is 
a  good  thing  to  do  and  someone  [first  source]  will  praise  [first 
reinforcement]  you,  but  someone  else  [second  source]  will  scold  [second 
reinforcement]  you?"  Complex  questions  of  this  kind  were  not  tested  in 
Project  ECHO,  but  seem  well  worth  testing. 

Complexity  and  Confusion 

A  complex  question  that  was  used  in  Project  ECHO  was:  "What  is  a 
good  thing  that  has  happened  [past  event],  or  could  happen  [possible 
future  event],  to  a  person  like  you?"  (de  Mille  and  Barthol,  1969). 

This  question  was  meant  to  cover  a  wide  range,  and  it  did.  However,  the 
combination  of  the  two  kinds  of  events  on  the  same  question  card  resulted 
in  data  that  were  difficult  to  analyze,  because  most  ^s  did  not  specify 
which  of  the  event  questions  they  were  answering,  has-happened ,  or  could- 
happen.  The  _E  concluded  that  it  would  have  been  better  to  present  each 
^  with  five  question  cards  asking  the  past-event  question  and  five  more 
asking  the  possible-event  question,  rather  than  ten  cards  asking  a 
complex  question. 


42 


Going  back  to  the  praise-and-scold  complex  question,  confusion 
could  be  avoided  by  stating  the  whole  complex  question  and  then  following 
it  with  three  separate  questions,  each  with  a  space  for  writing  the 
answer: 

"What  is  the  good  thing?" 

"Who  would  praise  you?" 

"Who  would  scold  you?" 

The  LCDS  question  set  elicits  complex  data  that  are  fully  analyzable 
because  adequately  separated. 

2 .  Repetitions  of  the  ECHO  Questions 

Bavelas  (1942)  presented  his  question  to  ^s  nine  times,  with  pro¬ 
gressive  intensification  after  the  third  and  sixth  times.  Most  ECHO 
studies  have  used  ten  cards  for  each  question,  and  most  ^s  have  filled 
out  all  ten  cards.  To  discover  whether  fewer  repetitions  would  yield 
equivalent  data,  four  samples  were  analyzed  (see  Appendix  II,  Manila 
Study) .  The  analysis  indicated  that  95  percent  of  the  variance  in  the 
size  of  value  categories  (where  size  is  a  function  of  the  number  of  ^s 
represented  in  the  category)  was  determined  by  the  first  seven  cards 
filled  out  by  The  reduction  of  the  sample  to  seven  cards  per  question 
resulted  in  a  trivial  loss  of  categories;  the  group  percentages  repre¬ 
sented  in  the  categories  were  slightly  lower. 

The  E  concluded  that  7-card  question  sets  were  satisfactory  when 
the  number  of  Ss  was  approximately  30  or  more;  that  group  percentages 
from  10-card  sets  should  not  be  compared  with  those  from  7-card  sets; 
that  it  was  advantageous  to  increase  the  number  of  ^s  rather  than  the 
number  of  cards  per  S_;  and  that  the  7-card  set  was  generally  preferable 

:k 

to  the  10-card  set. 

When  the  LODS  question  set  is  used,  it  is  desirable  to  reduce  ^  fatigue 
by  limiting  the  number  of  cards  per  question  to  seven.  Since  seven  Like 
and  seven  Ought  questions  are  a  rough  equivalent  of  14  good  questions, 
the  equivalent  data  collected  from  each  ^  are  actually  increased  by  using 
the  14-card  LO  (Like,  Ought)  set  instead  of  the  10-card  good  set. 
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3. 


Type  of  Data  Collection 

Four  ways  of  collecting  answers  to  ECHO  questions  have  been  listed: 
group  administration,  individual  interview,  self-administered  questions, 
and  data  collection  by  mail. 

Group  Written  Administration 

This  technique  is  preferable  to  the  other  three  because:  it  is 
quicker,  cheaper,  and  easier;  ^  motivation  is  more  reliable;  fewer 
packets  are  returned  uncompleted;  control  over  extraneous  variables  is 
greater;  and  the  data  are  more  comparable  to  other  samples  collected  in 
the  same  way. 

Individual  Oral  Interviews 

Because  some  prospective  ECHO  ^s  cannot  read  or  write  well  enough 
to  understand  written  instructions  and  write  answers  to  questions  (for 
example,  peasants  in  Northeast  Thailand) ,  some  ECHO  data  have  been 
collected  in  individual  oral  interviews.  The  additional  costs  of  this 
technique  are  in  the  need  for  a  skilled  interviewer  and  in  the  time 
required  for  each  interview. 

Barthol  and  Bridge  (1967)  reported  that  ^s  had  taken  1  hr  20  min 
to  give  10  good  and  10  bad  responses.  They  attributed  this  lengthy  time 
not  to  the  difficulty  of  the  task  for  illiterate  ^s  but  to  the  ^s  * 
desire  to  talk  to  someone  who  was  willing  to  listen.  In  the  East  Los 
Angeles  Skill  Center  Study  (see  Appendix  II) ,  oral  interviews  took  an 
hour  on  the  average. 

In  the  Vietnam  Study  (Bridge  and  Heller,  1968)  oral  interviews 
were  compared  with  group  written  administration,  and  two  methods  of 
recording  oral  interviews  were  compared.  Forty-seven  Vietnamese  soldiers 
were  interviewed;  approximately  half  of  the  interviews  were  tape 
recorded,  and  the  other  half  recorded  in  writing  by  the  interviewer.  The 
data  were  classified  together  into  a  common  system  and  the  two  pairs  of 
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resulting  distributions  compared.  The  tape  recording  method  apparently 
did  not  affect  the  data  in  any  important  way;  a  significant  Spearman  rank 
correlation  of  +0,83  was  found  between  the  hierarchies  of  whose  oral 
responses  were  taped  and  those  whose  responses  were  written  down. 

The  also  compared  the  oral  interview  data  and  the  standard 
written  data  in  the  Vietnam  Study,  obtaining  a  significant  rank  correla- 
tion  of  +0.71.  This  correlation  is  about  the  same  size  as  correlations 
between  US  universities  or  between  males  and  females  within  the  same 
group  (see  Appendix  II,  Intercultural  College  Study);  those  value  orderings 
are  indeed  quite  similar,  but  a  few  large  specific  differences  do  exist. 

For  example,  the  Howard  male-female  correlation  was  +0.70,  yet  28  percent 
of  the  males  but  nn  females  were  represented  in  the  good- thing  category, 
'Have  sexual  relations'.  Such  disappearance  of  a  category  is  the  chief 
hazard  in  using  the  oral  interview,  where  anonymity  is  not  likely  and  the 
respondent  may  believe  that  the  interviewer  is  judging  him.  Had  the 
Howard  female  data  been  obtained  by  oral  interview,  the  investigators 
would  have  suspected  that  no  females  were  represented  in  the  category 
precisely  because  of  the  oral  interview. 

The  oral  interview  is  a  costly  way  of  eliciting  value  orderings 
that  are  likely  to  be  similar  to  those  obtained  with  the  written  instru¬ 
ment  but  from  which  whole  categories  may  be  missing  or  otherwise 
drastically  changed.  A  great  chance  exists  also  that  some  irrelevant 
characteristic  of  the  interviewer,  such  as  hair  color,  age,  name,  sex, 
or  tone  of  voice,  may  bias  the  ^'s  response.  The  E  concludes  that, 
though  ECHO  data  can  successfully  be  collected  by  oral  interview,  the 
written  form,  wherever  applicable,  is  much  preferred. 

Self-Administered  Questions 

When  ^s  are  not  in  the  presence  of  an  administrator  while  answering 
the  ECHO  questions,  their  motivation  may  be  less,  and  distractions  may 
be  more  likely.  The  lack  of  oral  instructions  makes  non-compliance  with 
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the  written  instructions  more  likely;  Bridge  and  Heller  (1968)  concluded 
that  errors  were  more  frequent  with  self-adminis tered  data  collection. 

In  the  Arrowhead  Training  Group  Study  (Appendix  II) ,  the  ^  found  that 
10  percent  of  the  _Ss  did  not  complete  their  self-administered  cards. 

Data  Collection  by  Mail 

In  addition  to  the  faults  just  attributed  to  the  self-administered 
data  collection,  data  collection  by  mail  has  the  drawback  that  many  ^s 
do  not  return  their  packets  at  all.  Mail  samples  tend  to  be  composed  of 
volunteer  _Ss  and  therefore  to  be  less  comparable  to  other  kinds  of  samples. 
Nevertheless  5  in  comparing  ECHO  responses  in  a  group  that  first  had  self¬ 
administration  and  then  mail-administration  (Arrowhead  Training  Group 
Study),  the  E  could  find  no  difference  in  the  self-administered  data 
between  returners  and  non-returners. 

Only  48  percent  of  the  Arrowhead  group  returned  their  packets  when 
surveyed  by  mail.  Bridge  and  Heller  (1968)  sent  packets  to  Vietnamese 
civilians  through  Vietnamese  intermediaries  and  reported  a  73-percent 
return. 

The  JE  concludes  that  data  collection  by  mail,  because  it  is  com¬ 
paratively  cheap  and  convenient,  is  a  useful  procedure,  though  such  data 
cannot  be  compared  with  confidence  to  data  collected  in  other  ways. 

4 .  Sample  Size 

Barthol  and  Bridge  (1967)  reported  that  data  from  subsamples  of  15 
or  20  people  accurately  reflected  the  value  hierarchy  of  a  total  sample 
of  100  or  200  ^s .  They  concluded  that  a  very  small  ECHO  sample  can 
yield  a  useful  picture  of  the  major  values  in  a  group,  but  went  on  to  say 
that  when  the  E  wishes  to  pinpoint  values  lower  in  the  hierarchy,  larger 
numbers  of  Ss  are  needed. 
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Samples  in  the  ECHO  project  varied  from  five  to  around  200  ^s. 
Many  of  the  larger  samples  were  not  homogeneous  and  were  partitioned  into 
subsamples.  Homogeneous  samples  of  30  to  60  ^s  were  found  to  yield  very 
satisfactory  information. 

5.  Lower  Age  Limits 

Bavelas  (1942)  reported  that  children  four  years  old  could  answer 
his  orally  presented  questions  but  that  the  fourth-grade  level  was 
appropriate  for  written  response.  Barthol  and  Bridge  (1967)  found  that 
children  eight  years  old  could  write  ECHO  answers  on  IBM  cards. 

6 .  Lower  Educational  Limits 

Illiterate  and  semiliterate  ^s  can  respond  to  orally  presented 
ECHO  questions,  and  there  would  seem  to  be  no  lower  educational  limit  for 
such  presentation.  In  the  East  Los  Angeles  Skill  Center  study  (Appendix 
II) ,  10  of  the  30  Ss  who  wrote  ECHO  answers  had  not  received  education 
beyond  the  primary  grades,  and  the  data  proved  satisfactory. 

Lower  educational  limits  for  classifiers  involve  the  need  to  read 
what  is  written  on  response  cards.  In  the  East  Los  Angeles  Skill  Center 
study,  classifiers  who  had  not  gone  beyond  the  sixth  grade  produced 
categories  that  were  useful  to  staff  classifiers.  On  the  other  hand, 
when  indigenous  classifications  are  to  be  interpreted  without  modifica¬ 
tion,  higher  levels  of  education  are  desirable. 

7 .  Data  Collection  Time  Requirement 

Adult  ^s  who  can  read  the  instructions  with  no  difficulty  can 
complete  20  question  cards  in  20  to  35  minutes.  Third  graders  have  taken 
1-1/2  hr  to  do  the  same  thing.  Oral  interviews  typically  require  an  hour. 

8.  Number  of  Responses  to  be  Classified 

Barthol  and  Bridge  (1967)  found  that  the  optimum  number  of  cards 
for  untrained  classifiers  to  handle  in  one  session  was  250  to  300. 
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Larger  samples  can  be  classified  by  breaking  the  session  up  into  two  or 
more  periods.  Trained  classifiers  can  handle  larger  numbers  of  cards 
without  fatigue. 

E.  APPLICABILITY  OF  THE  ECHO  METHOD 

* 

Table  2  shows  a  variety  of  samples  collected  during  Project  ECHO. 
One  of  the  main  purposes  dictating  the  selection  of  these  different 
groups  was  the  need  to  test  the  applicability  of  the  ECHO  method  in 
various  populations  and  to  various  kinds  of  problems.  This  section 
reports  on  the  problem  of  translation  into  foreign  languages;  the  pro¬ 
ductivity  of  ECHO  in  foreign  cultures,  in  special  domestic  subcultures, 
in  industrial  settings,  and  in  training  programs;  and  the  ability  of 
ECHO  to  discriminate  between  groups. 

1.  Translation  of  ECHO  Questions  and  Instructions 

ECHO  questions  and  instructions  for  question  and  classification 
sessions  have  been  translated  successfully  into  Spanish  (Barthol  and 
Bridge,  1967;  Appendix  II,  ELASC  Study),  Thai  (Appendix  II,  Thai  Study), 
and  Vietnamese  (Bridge  and  Heller,  1968).  The  translation  of  "good" 
into  Thai  illustrates  the  need  for  painstaking  evaluation  of  the  equiva¬ 
lence  of  terms  in  different  languages;  the  solution  was  the  LODS  question 
set. 


2.  ECHO  Productivity  in  Foreign  Cultures 

ECHO  has  elicited  analyzable  and  interpretable  data  from  Cuban 
refugees  (Barthol  and  Bridge,  1967),  Spanish  speaking  occupational 
trainees  in  the  United  States  (Appendix  II,  ELASC  Study),  students  in 


Missing  from  this  list  are  319  Vietnamese  soldiers,  and  civilian  men, 
women,  and  children  surveyed  by  Bridge  and  Heller  (1968);  and  222 
similarly  mixed  Vietnamese  surveyed  by  Lcdr  R.A.  McGonigal,  USN,  who  read 
about  ECHO  in  an  early  report  and  volunteered  to  have  his  interview  team 
collect  ECHO  data  (before  an  ECHO  team  had  gone  to  Vietnam). 
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TABLE  2 


POPULATIONS  SAMPLED  IN  ECHO  RESEARCH 


Sample  N 

Third  Grade  Pupils  17 

Sixth  Grade  Pupils  24 

Hoover  High  School  Students  112 

Oakland  High  School  Students  180 

Stanford  University  Students  72 

University  of  California,  Los  Angeles,  Students  137 

Northwestern  University  Students  68 

Howard  University  Students  60 

Manila  College  Students  66 

Kenya  Students  60 

Thai  Students,  University  of  California,  Los  Angeles  5 

Thai  Students,  University  of  Southern  California  23 

Thai  Students,  Thailand  41 

Thai  Interviewers,  Thailand  12 

Cuban  Exiles  43 

Spanish  Speaking  Occupational  Trainees,  East  Los  Angeles 

Skill  Center  45 

Student  Council  Training  Group,  University  of  California 

Santa  Barbara  18 

Arrowhead  Training  Group  90 

High  School  Training  Group  50 

University  Training  Group,  Leeds,  England  13 

Sorority  Members  30 

Fraternity  Members  35 

Nurses  52 

Dairy  Company  Employees  54 

Insurance  Company  Employees  93 

Airline  Personnel  44 

Tracturing  Company  Employees  (Pentalith)  105 
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Kenya  and  Manila  (Appendix  II) ,  Thai  students  and  interviewers  (Appendix 
II),  and  Vietnamese  soldiers  and  civilian  men,  women,  and  children 
(Bridge  and  Heller,  1968).  The  richness  and  interpretability  of  the  Thai 
data  suggest  that  ECHO  can  be  used  to  advantage  in  Thailand. 

3.  ECHO  Productivity  in  Special  Domestic  Subcultures 

An  unsuccessful  attempt  was  made  to  collect  data  from  disadvantaged 
black  young  men,  many  of  whom  were  thought  to  be  covertly  hostile  to  the 
data  collection  (Appendix  II,  Watts  Study).  This  failure  (due  partly  to 
illiteracy  in  the  target  group)  draws  attention  to  the  need  for  special 
techniques  for  collecting  ECHO  data  from  hostile  groups.  The  need  is 
pressing,  since  it  would  be  advantageous  to  understand  the  value  systems 
of  such  groups. 

4.  ECHO  Productivity  in  Work  Settings 

ECHO  studies  were  made  of  dairy  company  employees  (Barthol  and 
Bridge,  1967),  insurance  company  employees  (Barthol  and  Bridge,  1968), 
airline  employees,  and  "tracturing”  company  employees.  All  studies 
yielded  analyzable  and  interp re table  data.  The  findings  in  the 
” tracturing”  company  study  were  reported  to  the  company  at  length 
(de  Mille  and  Barthol,  1969)  and  were  confirmed  by  company  executives. 

5 .  ECHO  Measurement  of  Changes  Due  to  Training 

Four  attempts  were  made  (Appendix  II:  Student  Council  Study, 
Arrowhead  Training  Group  Study,  High  School  Training  Group  Study,  and 
Leeds  Study)  to  measure  the  effects  of  brief  or  extended  training  pro¬ 
grams.  Practical  difficulties  prevented  the  completion  of  any  of  these 
studies  in  the  original  form,  and  the  ability  of  ECHO  to  measure  changes 
due  to  training  is  still  untested. 

6 .  ECHO  Discriminations  Between  and  Within  Groups 

The  development  of  staff  classification  of  multigroup  data  and  the 
adoption  of  a  better  unit  of  observation  (one  ^  represented  in  a  category. 
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by  one  or  more  responses)  led  to  the  development  of  advanced  computer 
programs  for  analyzing  ECHO  data  (see  Appendix  III) .  Using  these  pro¬ 
cedures,  the  ability  of  ECHO  to  discriminate  in  detail  (and  statistically 
evaluate  the  differences)  between  groups  from  different  cultures 
(Appendix  II,  Intercultural  College  Study),  between  groups  within  the 
same  culture  (de  Mille  and  Barthol,  1969),  and  between  logically  defined 
subgroups  (males  and  females)  was  clearly  demonstrated. 


In  addition  to  the  ability  to  discriminate  between  subgroups  that 
are  logically  defined  (for  example,  male  and  female,  older  and  younger), 
it  has  been  suggested  that  ECHO  should  be  able  to  discover  unsuspected 
subgroups  in  a  sample  originally  thought  to  be  homogeneous.  Such  an 
ability  would  add  substantial  informational  utility  (through  discovery  of 
unexpected  information)  to  an  instrument  already  endowed  with  that  quality. 
During  Project  ECHO,  the  idea  was  conceived  that,  by  correlating  individual 
^s  with  each  other  (across  their  representations  in  categories),  ^s  of 
different  types  could  be  clustered  together.  The  appearance  of  two  or 
more  mutually  exclusive  clusters  in  a  group  would  be  strong  evidence  that 
the  sample  was  not  homogeneous  but  contained  two  or  more  distinct  subgroups. 


Because  of  the  irregularity  of  representation  in  categories  (an 
^  is  seldom  represented  in  all  categories),  the  usual  assumptions  for 


Kalhorn  (1944)  proposed  the  as  the  unit  of  observation,  but  Havighurst 
and  Neugarten  (1955)  went  back  to  the  response  as  the  unit  of  observation. 
Since  counting  ^-representations  is  more  difficult  than  counting  responses, 
whether  manually  or  by  machine,  the  need  to  switch  to  the  new  unit  in  the 
Pentalith  Study  (de  Mille  and  Barthol,  1969)  brought  the  computer  pro¬ 
grams  UNIKOUNT  and  PERZPROB  into  existence.  In  that  study,  the  rank 
ordering  of  value  categories  by  response  count  was  correlated  +0.90 
(Pearson  r)  with  the  rank  ordering  by  ^-representation.  Though  the  new 
unit  of  observation  made  relatively  little  difference  in  the  hierarchy  of 
values,  it  added  needed  power  to  the  statistical  analysis  while  eliminating 
distortions  caused  by  isolated  one-track  _Ss  whose  responses  fell  into  only 
one  or  two  categories,  which  were  thereby  inflated. 
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inter-^  correlation  do  not  hold;  special  calculations  would  have  to  be 
worked  out  and  tested.  The  problem  was  so  stated  during  Project  ECHO, 
but  further  inquiry  was  deferred. 


F.  EVALUATION  OF  THE  ECHO  METHOD 

Milburn,  Barthol,  and  de  Mille  (1968)  described  and  discussed  four 
criteria  by  which  methods  of  psychological  value  study  may  be  evaluated: 

Reliability:  The  tendency  of  a  measuring  instrument  or 

observation  technique  to  give  consistent  readings 
or  information  under  equivalent  conditions 

Validity:  The  degree  to  which  measurements  or  observations 

represent  the  phenomena  they  are  supposed  or 
expected  to  represent 

Utility:  The  degree  to  which  the  cost/benefit  ratio  of 

an  instrument  or  procedure  is  favorable 

Informational 

Utility:  The  tendency  of  an  instrument  or  procedure  to 

return  unexpected,  useful  information 

These  authors  compared  ECHO  to  five  other  methods  (content  analysis, 
systematic  observation,  interview,  projective  techniques,  and  non-ECHO 
surveys)  and  concluded  that  ECHO  was,  on  the  available  evidence,  satis¬ 
factory  in  reliability,  not  apparently  deficient  in  validity,  satisfactory 
in  utility,  and  superior  in  informational  utility. 

In  this  section,  some  detail  will  be  added  to  the  discussion  of 
these  points. 


1. 


ECHO  Reliability 

The  reliability  of  an  ECHO  hierarchy  of  values  has  two  components: 

•  The  consistency  with  which  a  group  responds  to  the  ECHO 
questions 

•  The  consistency  with  which  the  responses  are  classified 
into  categories 
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Estimates  of  these  two  components  of  reliability  have  been  made  in 
different  ways. 

a.  Group  Response  Reliability 

Havighurst  and  Neugarten  (1955)  estimated  the  six-month  test-retest 
reliability  of  responses  from  60  boys  and  70  girls;  computing  across  65 
value  categories,  they  found  correlations  of  0.91  for  the  girls  and  0.86 
for  the  boys,  which  indicate  high  reliability.  In  Project  ECHO,  test- 
retest  reliability  was  estimated  in  the  Student  Council  Study  (Appendix  II), 
where  correlations  computed  across  19  to  30  categories  ranged  from  0.71 
to  0.84.  The  lower  estimates  in  the  ECHO  project  may  arise  from  two  kinds 
of  difference  between  the  studies:  quality  of  classification  and  intent 
to  change  values  by  training.  Havighurst  and  Neugarten ’s  data  were 
classified  by  the  ^s  into  many  presumably  well-formed  categories  (that 
included  good  and  bad  responses  together) ;  the  ECHO  data  were  classified 
into  fewer  categories  by  untrained  classifiers  (goods  and  bads  being 
separately  classified).  No  known  relevant  activity  intervened  between 
Havighurst  and  Neugarten’ s  two  data  collections,  whereas  the  ECHO  _Ss 
participated  in  a  training  program  related  to  values. 

Some  other  indicators  of  ECHO  group  response  reliability  are  the 
correlations  obtained  when  a  group  is  split  into  subgroups.  In  the 
Intercultural  College  Study  (Appendix  II) ,  males  and  females  were 
separated  and  correlations  between  them  computed  (across  the  same  147 
categories) .  Table  3  shows  the  intersex  correlations  for  the  three  US 
and  three  foreign  groups.  (These  are  conservative  estimates  of  relia¬ 
bility,  since  one  would  expect  greater  differences  between  males  and 
females  than  between  two  random  samples  from  a  homogeneous  group.  All  of 
the  correlations  are  significant.) 

A  similar  estimate  of  group  response  reliability  is  found  in  Bridge 
and  Heller’s  (1968)  significant  Spearman  rank  correlation  of  +0.83 
between  two  methods  of  recording  oral  data,  where  ^s  were  randomly 
assigned  to  the  two  methods. 
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TABLE  3 


CORRELATIONS  BETWEEN  MALE  AND  FEMALE  SUBGROUPS 


Sample  r 

University  of  California,  Los  Angeles  .83 
Northwestern  University  ,70 
Howard  University  .66 
Manila  ,81 
Kenya  . 70 
Thai  , 74 


b.  Classification  Reliability 

The  usual  way  of  classifying  ECHO  data  is  to  allow  the  classifiers 
to  develop  their  own  categories.  Two  teams  of  classifiers  are  likely  to 
have  somewhat  different  criteria  for  developing  categories  (a  useful 
difference  when  ^  is  looking  for  latent  categories) .  Inspection  of  the 
data  typically  shows  that  interteam  differences  in  the  breadth  of  cate¬ 
gories  account  for  most  of  the  apparent  differences  in  classification. 
Thus,  one  classification  team  develops  a  category  "Do  well  in  school," 
while  a  second  team  distributes  those  same  cards  into  three  categories: 
"Study  hard,"  "Get  good  grades,"  and  "Graduate."  It  is  clear  that  the 
classifiers  are  in  substantial  agreement  on  the  meaning  of  the  responses, 
but  the  technique  for  quantifying  this  agreement  is  lacking. 

By  using  a  structured  sort ,  in  which  the  category  titles  from  one 
classification  team  are  used  by  a  second  team,  the  reliability  of  sorting 
can  be  estimated.  The  second  team  sorts  the  same  cards  used  by  the  first 
team;  if  the  sorting  process  is  completely  reliable,  the  two  teams  will 
sort  the  cards  identically.  Structured  sorting  was  done  many  times 
during  the  second  research  period  (Barthol  and  Bridge,  1967),  and  the 
discrepancies  seldom  involved  more  than  three  or  four  responses  in  a 
hundred. 
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Two  conditions  exist  under  which  structured  sorting  may  result  in 
lower  estimates  of  reliability.  The  first  condition  occurs  in  a  large 
classification  (for  example,  the  Intercultural  College  Study,  with  its 
5000  good  and  5000  bad  responses)  that  has  some  closely  related  categories 
as  well  as  some  very  inclusive  categories,  where  even  well-trained  and 
skillful  classifiers  arrive  at  different  conclusions  if  they  classify 
independently;  in  such  a  case  it  is  useful  to  have  at  least  one  structured 
sort  to  learn  which  categories  are  unreliable.  The  second  condition  that 
may  result  in  lower  estimates  of  reliability  occurs  when  the  initial 
classification  has  been  inept  and  categories  are  carelessly  constructed. 

ECHO  classification  can  be  improved  to  meet  the  needs  of  the 
research  by  refining  categories  until  the  teams  of  classifiers  are  in 
agreement. 

2.  ECHO  Validity 

Milburn,  Barthol,  and  de  Mille  (1968)  discuss  three  kinds  of  validity 
that  are  applicable  to  the  ECHO  method:  construct  validity,  predictive 
validity,  and  concurrent  validity. 

a.  Construct  Validity 

Construct  validity  involves  the  relations  between  observations 
predicted  or  explained  by  theoretical  constructs  within  a  unified  theo¬ 
retical  system.  Studies  that  would  show  evidence  of  construct  validity 
were  beyond  the  scope  of  Project  ECHO. 

b.  Predictive  Validity 

Predictive  validity  is  tested  by  success  or  failure  in  predicting 
later  criterion  observations  from  earlier  measurements  arising  from  the 
instrument  to  be  validated.  Studies  that  would  show  evidence  of  ECHO 
predictive  validity  would  require  a  longitudinal  design  and  an  acceptable 
criterion  measure,  the  latter  being  difficult  to  establish.  No  predictive 
studies  were  conducted  during  Project  ECHO. 
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c.  Concurrent  Validity 

Concurrent  validity  involves  relations  between  the  instrument  to 
be  validated  and  other  instruments  whose  validity  is  established  (or 
assumed) .  Three  kinds  of  concurrent  validity  checks  were  used  in  Project 
ECHO:  ECHO  message  sessions,  informed  opinion,  and  an  alternate 

ins  trument . 

The  concurrent  validity  estimates  provided  by  message  sessions 
(Barthol  and  Bridge,  1967;  1968)  were  quite  varied,  depending  in  part  on 
the  fineness  of  discrimination  attempted  (for  example,  Cuban  exiles  versus 
Stanford  graduate  students,  an  easy  discrimination;  male  versus  female 
college  students,  a  difficult  discrimination)  and  the  relative  state  of 
development  of  the  message  session  technique  itself.  A  very  clear  and 
in terpre table  discrimination  was  observed  in  the  study  of  insurance 
company  employees  (Barthol  and  Bridge,  1968). 

Evidence  of  concurrent  validity  was  obtained  through  informed 
opinion  in  the  Kenya,  Manila,  and  Thai  studies  (Appendix  II).  Information 
based  solely  on  ECHO  findings  was  presented  to  persons  either  expert  or 
well-informed  about  the  indigenous  culture;  most  of  the  findings  were 
confirmed;  disagreements  were  few  and  minor.  In  the  Arrowhead  Training 
Group  Study  (Appendix  II),  ECHO  findings  were  confirmed  by  one  of 
the  professional  group  leaders.  In  the  Pentalith  Study  (de  Mille  and 
Barthol,  1969),  the  findings  were  presented  to  officials  of  the  company, 
who  confirmed  most  of  them  and  did  not  point  out  any  important 
diagreements . 

Also  in  the  Pentalith  Study,  evidence  of  concurrent  validity  came 
from  an  al t e rna te  ins t rumen t .  Five  opinion  poll  questions  were  presented 
to  the  ^s  along  with  the  ECHO  questions;  the  results  of  the  poll  questions 

JU 

and  the  ECHO  questions  were  in  complete  agreement. 

An  attempt  to  test  relationships  between  ECHO  categories  and  Rokeach 
value  statements  is  described  in  the  High  School  Training  Group  Study 
(Appendix  II) . 
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3.  ECHO  Utility 

Milburn,  Barthol,  and  de  Mille  (1968)  concluded  that  ECHO  utility 
was  probably  high,  depending  on  the  type  of  data  collection  (group  or 
individual)  and  the  size  of  the  population  in  relation  to  the  customarily 
small  ECHO  sample. 

In  comparison  to  other  methods  of  value  study  in  which  masses  of 
heterogeneous  data  must  be  analyzed  (for  example,  content  analysis),  the 
ECHO  classification  and  data  processing  techniques  have  relatively  high 
utility. 

In  any  kind  of  research,  some  costs  are  involved  in  the  effort  to 
make  the  findings  plausible  (as  distinct  from  correct) ,  and  cost  is  also 
involved  in  the  failure  to  make  the  findings  plausible.  When  ECHO  is 
compared  with  clinical  methods,  where  the  inferential  process  is  either 
hidden  from  or  incomprehensible  to  all  but  a  few  specialists,  ECHO  can  be 
seen  to  have  the  higher  utility,  since  the  entire  ECHO  process  can  be 
objectively  described,  and  all  of  the  data  can  be  displayed  suitably 
organized  for  interpretation  by  the  uninitiated  reader.  The  same 
comparison  would  hold  for  methods  of  value  study  in  which  masses  of 
heterogeneous  data  are  analyzed  but  techniques  are  not  available  for  dis¬ 
playing  the  organized  data  economically. 

4.  ECHO  Informational  Utility 

Specific  evidence  of  ECHO  informational  utility  came  from  the 
Arrowhead  Training  Group  Study  (Appendix  II),  where  an  unexpected  and 
useful  observation  about  one  of  the  training  groups  was  later  confirmed 
by  the  professional  group  leader.  In  the  Kenya  Study  (Appendix  II) ,  an 
essay  was  written  about  Kenya  from  classified  ECHO  responses;  it  contained 
many  items  of  true  information  about  Kenya  that  ^  had  not  expected  to 
discover.  These  two  illustrations  of  informational  utility  confirm  the 
common  sense  impression  that  an  instrument  that  elicits  spontaneous  and 
socially  relevant  ideas  must  contain  unexpected  and  useful  information. 
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G.  THEORETICAL  GROUNDWORK  AND  SPECIFIC  THEORETICAL  FINDINGS 

1 •  Theoretical  Groundwork 

The  relevance  of  the  ECHO  method  to  the  theoretical  field  of 
psychological  value  study  and  the  similarities  and  dissimilarities  of 
ECHO  to  some  other  methods  in  that  field  are  described  in  the  concurrent 
report,  The  ECHO  Method  and  the  Study  of  Values  (Milburn,  Barthol,  and 
de  Mille,  1968),  prepared  during  the  third  period  of  Project  ECHO.  That 
document  comprises  the  theoretical  groundwork  of  the  ECHO  method* 

2 .  Specific  Theoretical  Findings 

Project  ECHO  had  no  specific  theoretical  objectives,  since  effort 
was  mainly  devoted  to  developing  and  testing  the  method.  Nevertheless, 
there  were  theoretical  findings  (Barthol  and  Bridge,  1967;  1968; 

Appendix  II),  some  of  which  will  be  discussed  below. 

a.  Empirical  Relation  of  the  Concepts  "Good"  and  "Bad" 

Assuming  that  distinctions  between  good  and  bad  would  be  useful, 
Kalhorn  (1944)  analyzed  her  good  and  bad  data  separately;  the  same 
treatment  was  used  in  Project  ECHO.  Havighurst  and  Neugarten,  on  the 
other  hand,  took  the  position  that  usually  "no  psychological  distinction 
was  possible  between  the  positive  and  negative  mentions  of  a  category" 
(1955,  p.  91);  most  of  the  categories  they  produced  contained  logically 
opposite  answers  to  good  and  bad  questions. 

After  reviewing  the  literature,  Milburn,  Barthol,  and  de  Mille 
hypothesized  as  follows: 

It  appears  that  good  and  bad,  though  logical  opposites, 
are  not  often  opposites  on  linear  dimensions  of 
response,  but  are  either  scarcely  related  or  related 
in  a  non-linear  fashion  [1968,  p.  9]. 

During  Project  ECHO,  an  opportunity  arose  to  test  this  hypothesis.  The 

Intercultural  College  Study  (Appendix  II)  produced  a  list  of  67  categories 

This  analysis  is  not  reported  in  Appendix  II.  For  a  more  complete  treat¬ 
ment,  see:  R.  de  Mille ,  Logical  and  Empirical  Oppositeness  in  Value 
Responses,  Psychological  Reports , 1970 ^  26,  143-154. 
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of  good  things  to  do  and  80  categories  of  bad  things  to  do.  In  those  two 
lists  were  74  categories  whose  titles  declared  them  to  be  logical 
opposites  (as  examples,  Attend  Classes  Regularly  versus  Cut  Classes, 

Help  Parents  versus  Not  Help  Parents,  Get  Married  versus  Not  Marry,  Be 
Punctual  versus  Not  Be  On  Time,  Be  Clean  versus  Be  Dirty,  Be  Religious 
versus  Be  Irreligious  ) .  In  each  of  nine  college  groups ,  the  numbers  of 
S-representations  in  the  37  good  categories  were  correlated  with  those 
in  the  37  bad  categories.  The  correlation  coefficients  ranged  from  +0.01 
(Howard  University)  to  +0.57  (Manila).  Grouping  the  US  samples  together 
and  the  foreign  samples  together,  the  correlations  were  +0.30  (US)  and 
+0.43  (foreign).  The  correlation  between  good  and  bad  with  all  samples 
grouped  together  was  +0.17. 

It  should  be  noted  that  this  test  strongly  favored  the  logical- 
opposite  hypothesis  of  Havighurst  and  Neugarten,  since  the  other  73 
categories  were  excluded  precisely  because  no  logical  opposites  could  be 
found  for  them. 

The  results  appear  to  say  two  things: 

•  First,  groups  vary  considerably  in  the  degree  to  which 
their  values  and  dis values  are  correlated. 

•  Second,  though  values  and  disvalues  are  often  logical 
opposites,  they  should  not  be  assumed  to  be  empirical 
opposites;  the  evidence  suggests  that  values  and  disvalues 
are  largely  independent. 

b.  Felt  Importance  of  Values 

ECHO  categories  in  which  most  or  many  members  of  a  group  are 
represented  are  considered  to  be  important  to  the  group.  This  measurement 

The  paired  category  titles  may  be  found  in  Tables  5  and  6;  the  two 
members  of  each  pair  have  the  same  number  in  the  good-things  and  bad- 
things  lists.  The  category  numbers  of  the  paired  titles  are:  1,  3,  4,  5, 
6,  7,  10,  11,  15,  16,  20,  21,  22,  23,  24,  25,  30,  33,  34,  40,  44,  45, 

50,  52,  53,  55,  56,  57,  60,  61,  70,  71,  73,  74,  80,  81,  93. 
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of  importance  by  counting  people  (or  in  some  earlier  analyses,  by  counting 
responses)  is  quite  different  from  asking  ^s  to  rate  or  rank  values  for 
importance.  Barthol  and  Bridge  (1967)  found  that: 

•  When  individual  ^s  ranked  their  own  responses  for  impor¬ 
tance,  the  rankings  were  not  correlated  with  category 
ranks  based  on  number  of  responses. 

•  When  indigenous  classifiers  ranked  the  categories  for 
importance,  the  rankings  were  fairly  reliable  (+0.79)  but 
correlated  very  inconsistently  with  ranks  based  on  number 
of  responses;  the  correlations  obtained  were:  0.03,  0.18, 

0.36,  0.42,  and  0.83  (the  last  two  being  statistically 
significant) . 

Analysis  of  the  Student  Council  data  (Appendix  II)  resulted  in  four 
significant  correlations  between  importance  rankings  (by  indigenous 
classifiers)  and  ranks  based  on  number  of  ^s ;  the  correlations  obtained 
were:  0.39,  0.55,  0.68,  and  0.70. 

The  E  concludes  that  ECHO  category  ranks  computed  either  from  the 
number  of  ^s  or  from  the  number  of  responses  do  reflect  the  importance  of 
values  to  members  of  the  group.  Other  variables  (such  as  clarity  of 
definition  of  the  value,  or  lack  of  psychological  discomfort  in  talking 
about  the  value)  may  also  influence  the  rank  of  a  value;  such  variables 
were  not  studied  in  Project  ECHO. 

c.  Values  as  Group  Phenomena 

ECHO  is  assumed  to  measure  values  held  by  groups  rather  than  by 
individuals  or  small  subsets  of  individuals  in  a  group.  The  large  cate¬ 
gories,  in  which  most  of  the  ^s  are  represented,  seem  obviously  to  be 
group  values;  but  what  about  the  small  categories,  in  which  only  a  few 
^s  are  represented?  Are  they  group  values,  less  important  and  so 
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mentioned  by  fewer  ^s?  Or  are  they  values  peculiar  to  the  few  who 
mention  them? 

An  opportunity  to  investigate  this  problem  arose  in  the  Student 
Council  Study  (Appendix  II),  where  the  same  ^s  answered  the  ECHO  questions 
on  two  occasions.  The  ^  reasoned  that  if  the  small  categories  were  group 
values,  they  would  be  mentioned  by  different  sets  of  ^s  on  different 
occasions;  on  the  other  hand,  if  they  were  peculiar  to  individuals  or 
small  sets  of  ^s,  they  would  be  mentioned  by  the  same  ^s  on  both 
occasions . 

The  results  favored  the  first  hypothesis.  In  the  small  categories, 
about  half  of  the  composition  of  the  set  of  ^s  changed  from  one  occasion 
to  the  other.  The  E  concluded  that  small  categories  could  legitimately 
be  interpreted  as  group  values . 

The  assumption  that  values  are  held  in  common  in  homogeneous  groups 
could  be  further  tested  by  eliciting  good  and  bad  value  hierarchies  from 
a  homogeneous  group  (so  defined  by  some  of  their  nonvalue  characteristics) 
and  then  asking  the  same  ^s  to  rate  each  of  the  category  titles  on  a 
good-bad  scale.  The  hypotheses  would  be  as  follows: 

1.  Regardless  of  position  in  the  hierarchy,  values  will  be  rated 
good  and  disvalues  will  be  rated  bad  by  all  ^s . 

2.  Important  (high-ranking)  categories  will  be  rated  toward  the 
extremes  of  the  scale,  while  less  important  (low-ranking) 
ones  will  be  rated  toward  the  middle  of  the  scale. 

Support  of  the  first  hypothesis  would  confirm  the  assumption  that 
values  are  held  in  common  in  homogeneous  groups.  Slight  departures  from 
the  predicted  result  would  be  interpreted  as  error  attributable  to 

Results  congruent  with  this  assumption  were  obtained  by  Milburn  (personal 
communication)  in  a  study  where  ECHO  values  and  disvalues  were  rated  on 
semantic  differential  scales. 
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inaccurate  category  titles,  to  the  inclusion  of  a  few  atypical  ^s,  or  to 
some  other  cause.  Large  departures  would  be  interpreted  as  consequences 
of  a  failure  to  select  homogeneous  ^s  (so  defined  by  their  value 
responses)  or  as  dis confirmation  of  the  hypothesis. 

Support  of  the  second  hypothesis  would  confirm  the  conclusion, 
stated  in  the  previous  section,  that  the  size  of  a  value  category  reflects 
its  importance  to  members  of  the  group. 

d.  Socialization  Studies 

Barthol  and  Bridge  (1967)  hypothesized  that  children  in  the  sixth 
grade  would  mention  Self  (as  a  source  of  approval  and  disapproval)  more 
often,  and  Parents  less  often,  than  children  in  the  third  grade.  The 
results  in  a  study  of  41  children  supported  the  hypothesis.  The 
investigators  interpreted  the  change  as  a  sign  that  social  values  were 
being  internalized  between  the  third  and  sixth  grades;  a  possible  alter¬ 
native  interpretation  is  that  the  children  were  maturing  and  achieving 
greater  mastery. 

In  the  Sorority  Socialization  Study  (Appendix  II),  the  conclusion 
was  reached  that  the  values  of  new  sorority  members  became  more  like 
those  of  the  older  members  during  a  six-month  period,  while  the  values  of 
the  older  members  remained  stable. 

e.  Cultural  Descriptive  Findings 

Aspects  of  the  cultures  of  Thailand,  Philippines,  and  Kenya  are 
reported  in  the  Intercultural  College  Study,  Thai  Study,  and  Kenya  Study 
(Appendix  II) . 

* 

The  responses  of  deviant  ^s  are  often  recognized  as  deviant  by  the 
classifiers  and  put  into  a  miscellaneous  category,  where  they  would  be 
eliminated  from  a  test  of  these  hypotheses. 
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V. 


CONCLUSIONS 


This  chapter  gives  the  conclusions  reached  at  the  end  of  Project 

ECHO. 


A.  DEFINITION  AND  STATE  OF  DEVELOPMENT 

1.  Theoretical  Relevance 

The  ECHO  method  is  appropriate  to  the  theoretical  field  of 
psychological  value  study  though  not  necessarily  limited  to  that  field. 

2 .  Function 

The  method  generates  complex  information  about  patterns  of  value 
and  disvalue  in  relation  to  sources  of  social  reinforcement  or  to  causal 
agents.  These  patterns  are  verbally  expressed  by  respondents  in  a  group 
and  reflect  the  ideas  of  value  and  influence  that  prevail  in  the  group. 
Detailed  discriminations  between  groups  and  subgroups  in  the  same 
culture  or  in  different  cultures  can  be  made. 

3.  Viability 

As  described  in  this  report,  the  ECHO  method  is  ready  for  use  and 
can  be  applied  by  a  scientifically  trained  investigator. 


B.  EVALUATION  OF  METHOD 

1.  Reliability 

The  reliability  of  group  responses  to  the  ECHO  questions  is  satis¬ 
factory  for  use  in  both  theoretical  and  practical  studies.  The  relia¬ 
bility  of  the  ECHO  classification  technique  is  indeterminate,  but 
classifications  can  be  refined  and  stabilized  as  need  be  for  any  partic¬ 
ular  research  problem.  Estimates  of  the  reliability  of  value  hierarchies 
(combining  both  the  group-response  and  classification  components  of 


63 


reliability)  have  been  satisfactory.  The  reliability  of  source  hier¬ 
archies  is  very  high. 

2 .  Validity 

The  construct  validity  and  predictive  validity  of  ECHO  have  not 
been  tested;  no  unfavorable  evidence  has  been  observed.  The  concurrent 
validity  of  ECHO  (its  agreement  with  other  concurrent  measures  whose 
validity  is  established  or  assumed)  is  satisfactory. 

3.  Utility 

The  cost/benefit  ratio  of  ECHO  has  not  been  quantified.  However, 
ECHO  provides  much  information  from  relatively  small  and  economically 
surveyed  samples  of  respondents;  and  ECHO  techniques  of  data  analysis 
and  display  are  economical  in  comparison  with  earlier  techniques  for 
analyzing  and  displaying  such  complex  data. 

4 .  Informational  Utility 

ECHO  is  particularly  productive  of  unexpected  useful  information. 

C.  PROVEN  APPLICATIONS 

During  the  project,  ECHO  was  successfully  applied  in  the  following 
four  kinds  of  problems.  (It  is  not  suggested  that  ECHO  is  limited  to 
these  four.) 

1.  Foreign  Language 

The  ECHO  questions  and  the  instructions  for  question  sessions  were 
successfully  translated  into  Spanish,  Thai,  and  Vietnamese. 

2 .  Foreign  Cultures 

Analyzable  and  interpretable  data  were  collected  from  indigenous 
respondents  in  Thailand,  Kenya,  Philippines,  and  Vietnam. 
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3. 


Work  Settings 

Useful  data  were  collected  from  employees  of  four  different 
companies;  concurrent  validity  was  tested  and  demonstrated  for  two  of  the 
samples;  a  detailed  study  was  made  of  one  of  the  samples. 

4.  Educational  Settings 

Many  data  samples  were  collected  from  students;  third  grade,  sixth 
grade,  high  school,  college,  graduate,  and  professional  students  were 
surveyed. 

D.  APPLICATIONS  NOT  TESTED 

The  following  four  applications  of  the  ECHO  method  or  its  products 
were  considered  and  one  of  them  was  attempted,  but  none  of  them  were 
successfully  tested. 

1.  Training  Device 

The  use  of  ECHO  information  to  enhance  training  was  not  tested. 

2 .  Evaluation  of  Training 

Three  studies  were  designed  to  test  the  ability  of  ECHO  to  measure 
changes  due  to  training;  none  of  these  studies  could  be  completed  in  the 
original  form. 

3.  Informative  Communications 

The  usefulness  of  ECHO  information  for  generating  informative, 
discursive  communications  was  not  tested. 

4.  Persuasion 

The  usefulness  of  ECHO  information  for  directly  generating  per¬ 
suasive  messages  or  indirectly  enhancing  persuasive  techniques  was  not 
tested. 
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£•  METHODOLOGICAL  CONCLUSIONS 

1 .  Classification 

The  classification  technique  is  indispensable  to  the  ECHO  method. 

Both  indigenous  and  staff  classification  are  useful;  each  is  indispensable 
for  some  problems;  the  interaction  of  the  two  is  often  desirable. 

2.  Data  Collection  Techniques 

Answers  to  ECHO  questions  may  be  given  orally  to  an  interviewer, 
or  in  writing  in  a  group  session,  or  on  self-administered  forms  that  are 
handed  to  respondents  or  mailed.  Each  of  these  techniques  has  uses,  and 
corresponding  costs  and  limitations.  The  data  produced  by  the  different 
techniques  are  not  necessarily  comparable.  For  general  use,  the  group 
written  technique  is  preferred. 

3.  Sample  Size 

Homogeneous  samples  of  30  to  60  ^s  yield  satisfactory  information 
for  many  problems.  When  an  exhaustive  probing  of  values  is  desired, 
many  ^s  are  needed;  when  only  the  most  prevalent  items  are  of  interest, 
very  small  samples  (15-20)  may  be  useful. 

4.  Lower  Age  Limits 

Written  responses  have  been  collected  from  American  children  eight 
years  old;  oral  responses  have  been  collected  from  four-year  olds. 

5 .  Lower  Educational  Limits 

American  third-grade  children  have  written  answers  to  ECHO  questions. 
Spanish-speaking  classifiers  who  had  not  gone  beyond  the  primary  grades 
produced  categories  that  were  useful  to  staff  classifiers  though  not  good 
enough  to  justify  detailed  interpretation.  No  lower  educational  limit 
applies  to  oral  ECHO  responses. 


66 


6. 


Data  Collection  Time 


Adult  _Ss  who  can  read  the  instructions  with  no  difficulty  can 
complete  20  question  cards  in  20  to  35  minutes.  Third  graders  have 
taken  1-1/2  hours  to  do  the  same  thing.  Oral  interviews  typically  require 
an  hour. 

7.  Message-Session  Technique 

The  message-session  technique  gives  uneven  results,  possibly  owing 
to  unreliability  or  unrepresentativeness  of  the  category  titles,  which 
come  from  a  team  of  only  three  classifiers  rather  than  from  the  entire 
group  of  respondents. 


67 


VI 


RECOMMENDATIONS 


A.  RECOMMENDED  PROVEN  AND  POTENTIAL  USES  OF  THE  ECHO  METHOD 

The  first  three  recommended  uses  were  tested  in  Project  ECHO  and 
proven  effective.  The  other  six  uses  are  recommended  as  potential  uses 
that  have  not  been  tested. 

1 .  Theoretical  Research 

ECHO  should  be  used  to  advance  our  theoretical  understanding  of 
human  values  and  behavior. 

2.  Cultural  Description 

ECHO  is  recommended  as  an  economical  and  convenient  technique  for 
describing  the  value-and- influence  patterns  in  familiar  and  unfamiliar 
populations,  foreign  or  domestic,  and  for  the  discovery  of  many 
unexpected  and  useful  items  of  information  about  unfamiliar  cultures. 

3 .  Intergroup  Discrimination 

ECHO  can  discriminate  in  detail  between  groups  and  subgroups  in  the 
same  culture  or  in  different  cultures.  Differential  studies  can  be 
conducted  in  a  variety  of  settings,  including  work  and  educational 
settings . 

4 .  Behavior  Prediction 

It  should  be  possible  to  Improve  behavior  predictions  by  adding 
ECHO  variables  to  other  predictor  variables. 

5 .  Training 

ECHO  information  could  be  used  to  improve  training  for  field  work 
in  foreign  cultures  or  domestic  subcultures.  In  one  application,  ECHO 
information  from  the  selected  culture  could  be  added  to  the  indoctrina¬ 
tion  of  the  students.  In  another  application,  the  students  could  answer 
ECHO  questions  in  the  role  of  a  member  of  the  selected  culture,  or  could 
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be  given  the  responses  of  members  of  the  culture  to  classify,  and  the 
congruities  and  incongruities  between  student  responses  (or  categories) 
and  indigenous  responses  (or  categories)  could  be  used  as  instructional 
material.  Other  training  programs  where  values  were  relevant  could 
likewise  use  ECHO  information. 

6 .  Evaluation  of  Training 

Successive  ECHO  measurements  could  reflect  the  progress  of  student 
values  toward  congruity  with  the  values  of  members  of  the  selected 
culture  or  subculture.  Such  progress  would  constitute  increasing 
empathy  or  cross-cultural  understanding. 

7.  Communication 

ECHO  provides  indigenous  raw  material  that  should  be  useful  in 
constructing  discursive  informative  communications  to  members  of  other 
cultures  or  domestic  subcultures. 

8.  Persuasion 

ECHO  information  could  guide  the  construction  of  persuasive 
messages  by  revealing  what  values  should  be  included  or  praised,  what 
disvalues  excluded  or  deprecated,  what  sources  associated  with  what 
values,  and  what  favored  phrases  selected. 

9 .  Polling  Questions 

ECHO  should  improve  the  quality  of  polls  by  suggesting  specific 
questions  bearing  on  important  group  values  that  would  otherwise  be 
overlooked. 

B.  SPECIFIC  APPLICATIONS 

The  two  following  specific  applications  of  ECHO  are  recommended. 

1.  Thailand 

Findings  in  Project  ECHO  strongly  suggest  that  ECHO  could  be  used 
profitably  by  behavioral  scientists  to  learn  more  about  value  systems 
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in  rural  Thailand  and  to  develop  questions  and  hypotheses  that  have  not 
been  posed  by  previous  observers, 

2.  Community  Relations 

The  urgent  need  to  bring  all  useful  methods  to  bear  on  the 
domestic  disturbances  in  the  United  States  focuses  our  attention  on  the 
fact  that  lack  of  mutual  understanding  is  an  important  factor  in  such 
disturbances.  ECHO  should  be  useful  in  describing  the  value  systems  of 
the  opponents,  who  are  also  potential  collaborators  in  restoring 
domestic  peace.  ECHO  could  also  be  used  in  training  programs  related  to 
this  problem  (for  example,  training  for  work  with  disadvantaged  groups). 

C.  RECOMMENDED  METHODOLOGICAL  WORK 

Though  the  ECHO  method  has  been  developed  to  a  point  where  it  is 
readily  useable,  further  evaluation  is  needed,  and  certain  specific 
techniques  should  be  developed  or  improved. 

1.  Validity  Study 

Studies  of  construct  and  predictive  validity  should  be  conducted, 
and  further  evidence  of  concurrent  validity  should  be  collected. 

2.  Manipulation  of  ECHO  Parameters 

A  more  complete  understanding  of  the  instrument  should  be  achieved 
by  systematic  manipulation  of  all  of  the  ECHO  parameters  (for  example, 
role  of  the  respondent,  context,  complexity  of  questions). 

3.  Exploration  of  Background  Conditions  of  Values 

The  social  and  psychological  conditions  that  make  values  applicable 
or  inapplicable  should  be  studied. 

4.  Category  Titling  Procedure 

Because  of  the  suspected  weakness  in  category  titling,  procedures 
should  be  developed  for  increasing  the  reliability  of  category  titles. 
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5.  Data  Analysis  Techniques 

Further  work  should  be  done  on  data  analysis  techniques.  Specif¬ 
ically,  computer  programs  should  be  developed  for  correlating  cate¬ 
gories,  identifying  latent  categories,  and  correlating  or  clustering  ^s. 

6.  Hostile  Groups 

Techniques  are  needed  for  collecting  data  from  groups  that  are 
hostile  to  the  usual  procedures  or  personnel. 

7.  Informative  Communications 

Techniques  are  needed  for  using  ECHO  information  effectively  in 
constructing  informative  discursive  communications. 

8.  Persuasive  Messages 

Techniques  are  needed  for  using  ECHO  information  effectively  in 
guiding  the  construction  of,  or  supplying  material  for,  persuasive 
messages  and  programs. 
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APPENDIX  I 


AGENCY 

AGENT 

BADS 

CATEGORY  TITLE 

CLASSIFICATION 

DISPLAY  DECK 

DIS VALUE 

E 

ECHO  DATA  CARD 

EVENT 


GLOSSARY 


The  causation,  prevention,  or  modification  of  any 
event  by  any  agent,  as  described  by  ECHO  respondents; 
the  part  of  an  ECHO  question  that  specifies  agency 
and  elicits  responses  about  agents. 

Any  person,  entity,  or  force  that  ECHO  respondents 
describe  as  causing,  preventing,  or  modifying  an 
event. 

ECHO  questions,  responses,  or  categories  with  negative 
valuation  or  value,  such  as  "What  is  a  bad  thing  to 
do?" 

A  summary  statement,  or  label,  attached  to  a  set  of 
value  statements  or  source  responses  that  have  been 
grouped  into  one  category  in  an  ECHO  classification. 

An  ECHO  technique  in  which  responses  are  grouped  into 
emergent  or  inherent  categories  rather  than  sorted 
into  pre-existing  categories. 

A  deck  of  leaders  and  trailers.  The  display  deck 
makes  it  possible  to  display  all  responses  in  a  list, 
sorted  as  needed. 

A  specific  term  limited  to  the  negative  denotations 
of  the  term  value . 

A  symbol  for  the  experimenter,  investigator,  or 
session  administrator. 

A  question  card,  response  card,  leader,  trailer,  or 
reclassification  card. 

A  behavioral  or  nonbehavioral  occurrence  described 
by  ECHO  respondents;  the  part  of  an  ECHO  question 
that  elicits  this  description.  Behavioral  events 
are  usually  elicited  by  "to  do"  questions;  non¬ 
behavioral  events  by  "to  happen"  questions. 
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GOODS 


HIERARCHY 

IBM  CARD 
INDIGEN 

INDIGENOUS 

CLASSIFICATION 

INFLUENCE 

INFORMATIONAL 

UTILITY 

LEADER 

LODS  QUESTION  SET 

MESSAGE 

MESSAGE  SESSION 

N 

QUESTION  CARD 


ECHO  questions,  responses,  or  categories  with  positive 
valuation  or  value,  such  as  "What  is  a  good  thing  to 
do?" 

A  list  of  category  titles  in  their  order  of  importance 
for  a  group  of  ECHO  respondents.  The  most  important 
category  is  the  one  containing  responses  from  the 
most  group  members. 

An  electronic  data  processing  card. 

A  member  of  the  group  or  population  from  which  a 
particular  sample  of  ECHO  data  has  been  taken. 


Classification  done  by  a  team  of  indigens. 

Any  reinforcement,  reinforcer,  agency,  or  agent 
described  by  ECHO  respondents. 

The  tendency  of  an  instrument  or  procedure  to  return 
unexpected,  useful  information 

An  IBM  card  in  which  a  response  or  the  beginning  of 
a  response  has  been  keypunched. 

A  set  of  ECHO  questions  designed  to  elicit  statements 
about  things  the  respondent  would  like  (L)  to  do, 
ought  (0)  to  do,  would  not  like  (D)  to  do,  and  ought 
not  (S)  to  do.  The  symbols  D  and  S  can  be  remembered 
as  the  initials  of  dislike  and  shun. 

Usually,  a  short  list  of  high-ranking  (important) 

ECHO  value  category  titles  from  one  group  of 
respondents,  or  a  comparable  list  constructed  from 
some  other  information,  such  as  values  listed  by  an 
expert . 

An  ECHO  data  collection  in  which  a  group  of  Ss 
judge  alternative  messages  for  importance  or 
acceptability. 

A  symbol  standing  for  the  number  of  ^s  or  respondents 
in  a  group. 

An  IBM  card  with  ECHO  questions  printed  on  it, 
before  use  by  a  respondent  or  an  interviewer. 
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RECLASSIFICATION 

CARD 

REINFORCEMENT 


RELIABILITY 


RESPONSE  CARD 

ROLE 


S 

SOURCE 

SOURCE  RESPONSE 


An  IBM  card  on  which  a  response  has  been  printed  by 
computer  program  PRINDEK. 

The  positive  or  negative  reaction  of  others  to  one’s 
behavior,  as  for  example,  approval  or  disapproval, 
praise  or  blame;  the  part  of  an  ECHO  question  that 
specifies  the  reinforcement. 

The  tendency  of  an  instrument  or  observation  tech¬ 
nique  to  give  consistent  readings  or  information 
under  equivalent  conditions. 

A  question  card  after  use  by  a  respondent  or  an 
interviewer,  usually  bearing  a  written  response. 

The  point  of  view  that  the  respondent  takes  in 
answering  an  ECHO  question;  the  part  of  the  question 
that  specifies  the  point  of  view,  such  as  "a  person 
like  you,"  or  that  leaves  it  unspecified. 

A  symbol  for  a  respondent  or  subject  in  a  survey  or 
experiment.  Plural:  ^s . 

A  category  of  reinforcers  or  agents;  the  part  of  an 
ECHO  question  that  elicits  source  responses. 

A  verbal  response  to  an  ECHO  question  such  as  "Who 
would  approve?"  or  "Who  or  what  would  cause  it?" 


TRAILER 

UTILITY 

VALIDITY 

VALUATION 

VALUE 

VALUE  STATEMENT 


An  IBM  card  in  which  the  continuation  of  a  response 
has  been  keypunched. 

The  cost/benefit  ratio  of  an  instrument  or  procedure, 
particularly  a  favorable  ratio. 

The  degree  to  which  measurements  or  observations 
represent  the  phenomena  they  are  supposed  or 
expected  to  represent. 

The  part  of  an  ECHO  question  that  specifies  the 
goodness  or  badness  of  the  event. 

An  overt  or  implicit  (inferred)  response  that  attri¬ 
butes  goodness  or  badness  to  some  event  or  entity; 
a  category  of  value  statements,  with  their  title. 

A  verbal  response  to  an  ECHO  question  such  as,  "What 
is  a  good  thing  to  do?"  or  "What  is  a  bad  thing  to 
do?" 
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APPENDIX  II 


DETAILS  OF  SELECTED  STUDIES 


A*  INTRODUCTION 

The  purpose  of  this  appendix  is  to  provide  additional  details  about 
certain  studies  whose  findings  are  adduced  in  the  report  as  illustrations 
of  or  support  for  various  methodological  or  theoretical  points.  The 
appendix  is  organized  mainly  around  different  samples  of  data,  and  the 
sections  are  titled: 

INTERCULTURAL  COLLEGE  STUDY 
THAI  STUDY 
KENYA  STUDY 

EAST  LOS  ANGELES  SKILL  CENTER  STUDY 
STUDENT  COUNCIL  STUDY 
MANILA  STUDY 

OAKLAND  HIGH  SCHOOL  STUDY 
ARROWHEAD  TRAINING  GROUP  STUDY 
HIGH  SCHOOL  TRAINING  GROUP  STUDY 
SORORITY  SOCIALIZATION  STUDY 
INCOMPLETE  STUDIES 

The  order  in  which  these  studies  are  presented  is  somewhat  arbitrary, 
because  the  data  were  used  for  several  purposes,  most  of  which  are  not 
clearly  reflected  in  the  titles.  The  ten  completed  studies  do  not 
exhaust  all  of  the  activities  of  the  work  period,  November  1967  to 
December  1968,  but  represent  the  most  important  work  accomplished  during 
the  period  that  is  not  reported  elsewhere  (see:  Barthol  and  Bridge,  1968; 
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Bridge  and  Heller,  1968;  de  Mille  and  Barthol,  1969;  Milburn,  Barthol 
and  de  Mille,  1968;  and  Appendix  III  of  this  report).  The  work  done 
during  the  period  January  to  October  1967  was  reported  by  Barthol  and 
Bridge  (1967). 

B.  INTERCULTURAL  COLLEGE  STUDY 

1.  Background 

This  study  came  about  because  of  three  facts: 

1.  ECHO  data  that  had  been  collected  from  various  US  colleges 
during  earlier  project  work  were  available  for  additional 
analysis . 

2.  The  E  had  been  able  to  collect  data  from  Thai  Ss  while  on  a 
research  planning  trip  to  Thailand. 

3.  Unexpected  opportunities  arose  to  collect  data  in  Kenya  and 
the  Philippines. 

2 .  Objectives 

The  study  had  the  following  objectives: 

1.  To  provide  methodological  findings. 

2.  To  provide  a  context  in  which  the  Thai  Study  (reported  in 
Sec.  C,  herein)  could  be  given  cross-cultural  meaning. 

Both  of  these  objectives  were  achieved. 

3.  Sub j  ects 

Table  4  shows  the  number,  sex,  age,  and  grade  of  the  respondents 
in  each  group.  Each  sample  had  some  distinguishing  feature.  The 
Stanford  class  was  all  male  and  consisted  of  first-year  graduate  students 
in  business  administration;  the  UCLA  class  was  mixed  male  and  female 
undergraduates  (third  and  fourth  year)  in  a  psychology  class;  the 
Northwestern  class  was  all  first-year  undergraduates  in  a  psychology 
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TABLE  4 


COLLEGE  STUDY  SAMPLES 


N 

Male 

Female 

Age 

Year  (Class 

Stanford 

48 

48 

- 

22-30 

5 

UCLA* 

100 

50 

50 

19-25 

3-4 

Northwestern 

60 

30 

30 

18-19 

1 

Howard 

49 

15 

34 

17-22 

1-4 

Manila 

66 

27 

39 

17-21 

1-2 

Kenya 

60 

40 

20 

16-35 

Mixed 

Thailand 

(Computer  Class) 

26 

14 

12 

23-38 

5 

(Psychology  Class) 

15 

5 

10 

20-24 

2 

** 

USC-Thai 

23 

18 

5 

20-24 

2-4 

Somchai~Thai 

12 

12 

— 

25-30 

4+ 

University  of  California,  Los  Angeles. 

Thai  students  studying  at  the  University  of  Southern  California. 
Thai  field  interviewers  in  Thailand,  all  college  graduates. 
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class  and  of  both  sexes;  Howard  students  were  all  Negro;  the  Manila 
sample  consisted  of  two  college  classes,  one  all  male,  the  other  all 
female;  the  all-black  Kenya  sample  included  students  in  a  psychology 
class;  the  Thailand  sample  consisted  of  two  classes,  one  a  second-year 
psychology  class,  the  other  a  fifth-year  education  class  studying 
computer  operation;  the  USC-Thai  sample  consisted  of  Thai  students  in  the 
United  States;  and  the  Somchai-Thai  were  an  interviewing  team  in  Thailand, 

Three  of  the  groups  had  large  numbers  of  ^s ,  so  it  was  possible  to 
make  them  more  homogeneous  by  eliminating  atypical  ^s .  The  Stanford 
sample  was  reduced  to  48  by  eliminating  the  foreign  students  and  the  only 
female;  the  UCLA  sample  was  reduced  to  50  males  and  50  females  by  elimi¬ 
nating  students  over  26  years  of  age  and  freshmen  and  sophomores;  the 
Northwestern  sample  was  improved  statistically  by  excluding  one  graduate 
student,  five  sophomores,  and  two  who  were  older  than  19,  to  bring  that 
sample  to  30  males  and  30  females,  all  freshmen.  The  other  groups  were 
left  intact. 

4.  Sampling 

Customarily,  readers  are  warned  that  sampling  procedures  constrain 
the  generalization  of  a  study.  Although  the  sampling  was  non-random 
(the  foreign  classes  used  in  this  study  were  selected  because  they  were 
conveniently  available  to  colleagues  residing  in  Manila,  Kenya,  and 
Thailand) ,  the  practice  of  combining  subgroups  (such  as  Manila  males  and 
females,  or  Thai  computer  and  psychology  students)  increases  the  likeli¬ 
hood  that  the  findings  are  representative  of  the  broader  population  (such 
as  Manila  students  or  Thai  students).  If  a  phenomenon  is  found  in  each 
of  several  samples  of  one  population  or  not  in  any  sample  of  another 
population,  the  likelihood  that  the  finding  is  due  to  chance  decreases. 

In  this  study  the  populations  are  students  in  different  countries.  All 
results  reported  in  this  paper  describe  particular  groups  except  when 
the  results  are  specifically  identified  as  having  some  generality. 
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5. 


Method 


The  data  were  all  collected  in  the  same  way.  The  instructor  or 
E  read  the  standard  directions  to  the  class,  distributed  envelopes 
containing  the  ECHO  question  cards,  and  collected  them  when  _Ss  had 
finished.  Biographical  information  (age,  sex,  class)  was  written  by  ^s 
on  the  envelopes.  The  ^s  were  assured  of  anonymity.  Approximately  two- 
thirds  of  the  UCLA  sample,  one-half  of  the  Stanford  sample,  and  all  of 
the  Northwestern  sample  were  instructed  to  answer  the  question  "What  is 
a  good/bad  thing  for  a  student  like  yourself  to  do?"  In  the  other 
samples,  the  role  assignment  was  deleted.  (Inspection  of  the  US  quadrant 
of  Table  10  shows  that  the  role  assignment  had  little  if  any  differential 
effect  on  the  results.) 

The  data  were  first  classified  by  members  of  the  class  being  sur¬ 
veyed.  The  Thai  responses  (the  only  ones  not  given  in  English)  were 
translated,  and  English-language  responses  from  all  of  the  samples  were 
punched  into  display  decks  (see  Appendix  I) .  The  computer  program 
PRINDEK  (see  Appendix  III)  produced  uniform  cards  so  that  no  group  was 
identifiable  by  card  color  or  language.  All  cards  were  mixed  together, 
and  a  team  of  three  American  college  students  classified  the  approxi¬ 
mately  5000  goods  and  5000  bads.  Each  resulting  ECHO  category  was 
agreed  upon  by  the  classifiers  and  reviewed  and  refined  by  Another 

E  then  independently  reviewed  the  refined  categories,  and  the  two  ^s 
conferred  to  determine  the  final  classification. 

Category  numbers  were  assigned  so  that  similar  topics  or  similar 
value  clusters  were  contiguous.  Thus,  the  values  about  school  were 
assigned  category  numbers  1  through  9;  those  about  the  nation  and 
society  numbers  10  through  16,  and  so  on.  The  category  titles  appear 
in  Tables  5  and  6. 


K.  Bucknam,  H.  Goez,  and  M.  Gilson. 
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TABLE  5 


INTERCULTURAL  COLLEGE  STUDY  CATEGORY  TITLES 
Good  Things  To  Do 


1.  Study  hard  and  get  good  grades 

2.  Participate  actively  in  class  and  get 
to  know  the  instructors 

3.  Attend  classes  regularly 

4.  Obey  the  rules  and  regulations  of  school 

5.  Graduate  from  school  or  college 

6.  Go  to  graduate  or  professional  school 

7.  Be  educated  and  have  a  good  educational 
system 

8.  Get  and  use  information 

9.  Engage  in  extracurricular  activities 

10.  Be  loyal  to  and  serve  our  country 

11.  Follow  the  customs  and  obey  the  laws 
of  our  society 

12.  Do  something  useful  for  society  or  the 
community 

13.  Do  something  in  agriculture 

14.  Join  the  military  forces 

15.  Protest  against  the  ills  of  society 

16.  Be  loyal  to  and  serve  the  Black 
Movemen  t 

20.  Respect,  obey,  love  and  honor  our 
parents 

21.  Help  our  parents 

22.  Be  kind  to  our  parents 

23.  Love  and  take  good  care  of  family 
and  spouse 

24.  Be  kind  and  helpful  to  brothers  and 
sisters  and  other  relatives 

25.  Respect  and  obey  elders,  teachers  and 
superiors 

26.  Be  polite,  respectful  and  modest 

27.  Be  grateful  and  appreciative 

30.  Help  others 

31.  Help  the  needy 

32.  Help  others  educationally 

33.  Respect  the  rights  and  dignity  of  others 

34.  Love  and  forgive  one  another 

35.  Be  kind,  considerate,  sympathetic  and 

responsive  to  others 

36.  Be  honest,  sincere,  loyal  and  responsible 

37.  Be  generous  and  unselfish 

38.  Be  cheerful 


40.  Be  friendly  and  have  friends 

41.  Be  frank  and  open  with  others 

42.  Have  good  times  with  the  opposite  sex 

43.  Love  someone  and  be  loved 

44.  Have  sexual  relations 

45.  Get  married 

50.  Get  a  job  or  have  a  career 

51.  Work  hard,  do  your  best,  organize  your  time 
well 

52.  Be  punctual 

53.  Achieve  success  and  recognition 

54.  Be  independent  and  self-confident 

55.  Be  mature  and  self-controlled 

56.  Know  and  be  true  to  yourself 

57.  Stand  up  for  your  rights  and  convictions 

60.  Be  the  kind  of  person  who  can  live  life 
fully 

61.  Do  things  that  are  fun 

62.  Enjoy  quiet  recreation 

63.  Participate  in  sports  or  athletic  activities 

64.  Practice  creative  or  expressive  arts 

65.  Take  a  trip 

70.  Be  healthy  and  do  things  that  promote  good 
health 

71.  Keep  my  place  neat  and  clean 

72.  Lose  or  gain  weight 

73.  Be  attractive  or  correct  in  appearance, 
dress  and  grooming 

74 .  Make  and  save  money 

80.  Be  religious 

81.  Be  moral  and  good,  avoiding  vices  and  bad 
company 

90.  Decide  what  to  do  after  graduation 

91.  Drive  carefully 

92.  Get  a  drivers  license  or  a  car 

93.  Take  care  of  and  be  kind  to  pets  or  animals 

94.  Make  a  better  world 

95.  Make  this  a  better  country 

98.  Miscellany 
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TABLE  6 


INTERCULTURAL  COLLEGE  STUDY  CATEGORY  TITLES 
Bad  Things  To  Do 


1.  Get  bad  grades 

2.  Not  study  hard 

3.  Cut  classes 

4.  Violate  school  rules  and  regulations 

5.  Not  graduate  from  school  or  college 

6.  Not  go  to  graduate  or  professional  school 

7.  Be  uneducated  or  not  seek  education 

8.  Flunk  out  of  school 

9.  Cheat 

10.  Be  unpatriotic 

11.  Not  follow  the  customs  of  society 
13.  Misuse  the  resources  of  the  land 

15.  Conform  to  bad  customs  and  beliefs  of  The 
Establishment 

16.  Be  disloyal  to  the  Black  Movement 

17.  Be  a  social  dissident,  be  radical  or 
militant 

18.  Betray  one's  country 

19.  Break  the  law 

20.  Not  obey,  respect,  or  honor  parents 

21.  Not  help  parents  with  household  tasks 

22.  Be  unkind  or  inconsiderate  to  parents 

23.  Mistreat  or  neglect  spouse  or  children 

24.  Be  unkind  to  brothers,  sisters,  and  other 
relatives 

25.  Not  respect  or  obey  elders,  teachers,  or 
superiors 

26.  Not  get  along  well  with  other  people 

28.  Mistreat  one's  boy  or  girl  friend 

29.  Marry  prematurely  or  inappropriately 

30.  Not  help  others 

31.  Be  disloyal  or  untrustworthy 

32.  Exploit  or  manipulate  others 

33.  Not  respect  the  rights  or  dignity  of 
others 

34.  Hate  or  hold  a  grudge 

35.  Use  profane  or  obscene  language 

36.  Be  selfish  and  inconsiderate 

37.  Condemn,  judge,  or  criticize  others 

38.  Be  insincere  or  hypocritical 

39.  Look  down  on  others 

40.  Live  alone,  without  friends 

41.  Gossip  or  criticize  others  behind  their 
back 

43.  Feel  unworthy  or  unfortunate 

44.  Have  illicit  or  debasing  sexual  relations 

45.  Not  marry 


46.  Commit  adultery 

47.  Be  promiscuous 

48.  Sustain  or  cause  an  illicit  pregnancy  or 
birth 

50.  Be  socially  irresponsible  or  nonproductive 

51.  Not  try  hard  to  get  ahead 

52.  Not  be  on  time 

53.  Be  a  loser 

54.  Lie  or  be  dishonest 

55.  Be  careless  or  reckless 

56.  Be  foolish  or  self-deceptive 

57.  Not  act  according  to  one's  real  feelings  or 
best  convictions 

58.  Waste  time 

60.  Be  the  kind  of  person  who  cannot  live  life 
fully 

61.  Have  too  much  fun 

62.  Study  or  work  too  hard 

63.  Be  a  prostitute 

64.  Have  too  many  wives 

65.  Smuggle 

70.  Neglect  one's  health 

71.  Be  messy  or  dirty 

73.  Disregard  one's  clothing  or  personal 
appearance 

74.  Waste  money  or  run  up  debts 

75.  Gamble 

76.  Smoke 

77.  Drink  or  get  drunk 

78.  Take  drugs 

80.  Be  irreligious 

81.  Do  immoral  things  and  have  bad  companions 

82.  Hurt  people's  feelings  or  cause  trouble 
for  them 

83.  Injure  someone 

84.  Rape  someone 

85.  Kill  someone 

86.  Commit  suicide 

87.  Destroy  or  deface  others'  property 

88.  Steal 

93.  Hurt  or  kill  animals 

95.  Government  be  unfair  or  inept 

96.  Be  corrupt  in  government  service 

98.  Miscellany 
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Each  category  number  was  punched  in  the  cards  composing  that 
category.  All  cards  were  then  sorted  back  into  their  original  sample 
groupings.  Program  NUDEK  (see  Appendix  III)  then  merged  the  new  and  old 
information. 

The  _E  examined  lists  of  the  classified  responses  to  see  that  each 
was  logical  and  consistent.  To  check  the  appropriateness  of  the  US 
classification  of  foreign  data,  the  earlier  indigenous  classification 
codes,  also  listed,  were  used  as  a  reference.  When  the  Manila  data  were 
inspected,  using  the  indigenous  category  titles  to  determine  whether  the 
US  classifiers  had  assigned  the  same  meaning  to  the  responses,  only  one 
card  had  been  misclassified ,  and  two  of  the  three  cards  in  Miscellany 
might  have  been  placed  in  another  category. 

Inspection  of  the  Thai  data  showed  that  the  US  classifiers  under¬ 
stood  and  classified  all  but  three  cards  appropriately  according  to  their 
own  system  but  that  the  Thai  organization  of  the  data  was  sufficiently 
different  to  require  a  more  systematic  comparison.  Program  ROCKEM  (see 
Appendix  III)  was  used  for  this  purpose.  The  important  differences 
between  classifications  are  discussed  on  page  101  and  shown  in  Table  18 
of  this  appendix.  In  the  Kenya  data  sample,  inspection  of  each  card 
and  comparison  with  the  indigenous  classification  satisfied  ^  that  no 
gross  errors  had  been  made. 

The  classification  of  sources  of  approval-disapproval  and  the 
assignment  of  identification  numbers  was  performed  by  with  the  excep¬ 
tion  of  the  Manila  and  Thai  data,  where  virtually  all  source  assignments 
fit  the  US  system  and  only  trivial  adjustments  were  needed. 

Each  data  sample  was  processed  with  program  UNIKOUNT  to  obtain  ^ 
count  and  percentages  for  both  values  and  sources.  Each  group  was 
divided  by  sex  and  again  processed  by  UNIKOUNT,  so  that  inter-sex 
differences  could  be  evaluated  for  each  sample.  Program  PERZPROB 
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(see  Appendix  III)  was  used  to  compare  each  group  (or  subgroup)  quanti¬ 
tatively  with  every  other  group  (or  subgroup) .  Eight  principal  PERZPROB 
tables  were  generated:  group  (college)  comparisons  and  subgroup  (male- 
female)  comparisons  for  goods,  bads,  approval  sources,  and  disapproval 
sources . 

A  standard  computer  program  was  used  to  compute  inter-group  and 
inter-subgroup  correlation  coefficients  (Pearson  £)  .  A  standard  factor- 
analysis  program  was  also  used;  the  results  will  not  be  reported  but 
showed  quite  clearly  that  factor  analyzing  ECHO  data  should  be  useful, 
since  several  factors  (that  is,  US,  Thai,  Kenya,  and  Manila  factors)  that 
should  have  emerged,  did  so, 

6.  Results 


a.  Male-Female  Differences 

The  main  units  of  comparison  are  college  classes,  but  it  is  impor¬ 
tant  to  know  whether  any  of  the  observed  differences  might  result  from 
identifiable  subgroups.  An  obvious  partition  is  male-female.  Correlation 
coefficients  were  computed  between  the  male  and  female  subgroups  of  the 
UCLA,  Northwestern,  Howard,  Manila,  Kenya,  and  Thailand  samples.  Table  7 
shows  that  the  males  and  females  in  each  sample  responded  similarly  and 
that  the  data  can  safely  be  grouped  together  for  intercollege  comparisons. 

The  third  data  column  of  Table  7  incorporates  the  first  two  data 
columns.  The  correlation  coefficients  show  the  similarity  with  which 
males  and  females  from  the  same  colleges  responded  to  the  ECHO  questions. 
The  range  is  0.66  to  0.83.  The  coefficients  obtained  when  males  and 
females  are  compared  across  colleges  or  cultures  may  put  these  numbers 
in  perspective.  Thirty  other  pairings  are  possible  (for  example,  UCLA 
males  with  Northwestern  males,  Howard  females  with  Kenya  males);  of  these, 
only  five  correlations  were  as  high  as  or  higher  than  the  lowest  entry 
(0.61)  in  Table  7.  The  correlation  between  Thai  males  and  UCLA  females. 
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TABLE  7 


CORRELATIONS  BETWEEN  MALE  AND  FEMALE  SUBGROUPS 


Sample 


Goods 


Bads 


:k 

Toge ther 


UCLA 

.86 

.79 

.83 

Northwestern 

.72 

.70 

.70 

Howard 

.71 

.61 

.66 

Manila 

.90 

.69 

.81 

Kenya 

.65 

.74 

.70 

Thailand 

.82 

.68 

.74 

:k 

The  terms  ’goods’ 

and  ' b  ads ' 

are  used 

to  refer 

the  questions:  "What  is  a  good/bad  thing  to  do?" 
Correlations  in  the  last  column  are  computed  across 
goods  and  bads. 


for  example,  is  only  +0.32,  though  it  is  one  of  the  higher  cross-cultural, 
cross-sex  correlations. 

A  conclusion  is  that  male-female  value  orderings  are  similar  within 
cultures,  but  different  across  cultures.  Within  a  culture,  males  and 
females  are  about  as  much  alike  as  the  US  colleges  are  alike;  they  are 
much  more  alike  than  colleges  in  different  cultures  (see  Table  10). 

It  should  be  remembered  that  cross-cultural  study  may  obscure 
subtle  but  important  differences  within  cultures.  Intercultural  com¬ 
parisons  do  not  state  how  important  a  value  is  within  a  culture,  only 
that  it  exists  more  in  one  culture  than  in  another.  Similarly,  subgroup 
differences  are  overlooked.  The  value  categories  that  are  significantly 
different  (p  <  0.05)  when  males  are  compared  with  females  of  the  same 
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sample  are  shown  in  Tables  8  (goods)  and  9  (bads).  At  Northwestern,  for 
example,  females  held  higher  values  for  studying,  being  kind,  working 
hard,  being  independent,  and  knowing  one's  self;  the  males  were  higher  on 
success  and  sex.  At  UCLA,  females  were  higher  on  being  generous,  parti¬ 
cipating  in  the  arts,  losing  weight,  and  getting  a  driver's  license;  males 
were  higher  for  joining  the  military.  The  last  is  an  example  of  a  value 
that  is  sex-specific  though  not  widely  held  in  the  group. 

The  principle  for  studying  value  differences  between  college  groups 
is  exactly  the  same  as  for  differences  between  male  and  female  subgroups. 
Since  male-female  differences  do  exist  in  every  sample,  each  intergroup 
difference  was  checked  to  determine  whether  one  sex  or  the  other  had 
caused  the  difference  to  appear  between  groups.  In  the  few  instances 
where  sex  bias  was  a  factor,  the  lesser  intergroup  difference  (whether 
based  on  a  male  or  a  female  subgroup)  was  chosen.  The  intercultural 
differences  discussed  on  page  90  are,  therefore,  not  attributable  to 
sex  bias. 

b.  Intercollege  Correlations 

Table  10  shows  the  intercorrelations  among  the  nine  groups.  The 
coefficients  above  the  diagonal  were  computed  across  the  67  good  cate¬ 
gories;  those  below  the  diagonal,  across  the  80  bad  categories. 

Comparison  of  the  US  quadrant  with  the  unenclosed  portions  of  the 
table  shows  that  value  orderings  are  similar  among  the  US  colleges  but 
relatively  different  between  US  and  foreign  colleges.  Comparison  of  the 
two  (coincidentally  equal)  circled  numbers  with  their  respective  counter¬ 
parts  shows  that  Thai  students  in  Thailand  share  more  good  and  bad  values 
with  Thai  students  in  the  United  States  than  with  either  US  students  or 
other  foreign  students.  The  general  pattern  of  the  table  conforms  to 
what  would  be  expected  in  cross-cultural  value  research. 
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TABLE  8 


GOOD  TO  DO"  WITHIN-GROUP  COMPARISON  OF  MALES  AND  FEMALES 


MALES  HIGHER 

FEMALES  HIGHER 

Percentage 

Percentage 

Category  Idea 

M 

F 

Category  Idea 

M 

F 

UCLA 

14 

Join  military 

14 

0 

37 

Generosity  and 
unselfishness 

0 

8 

64 

Practice  creative  or 
expressive  arts 

2 

16 

72 

Lose  or  gain  weight 

4 

18 

92 

Get  a  driver's  license 

or  car 

0 

6 

NORTHWESTERN 

44 

Have  sexual  relations 

23 

0 

1 

Study  hard,  get  good 
grades 

37 

63 

53 

Achieve  success  and 
recognition 

23 

3 

35 

Kind,  considerate, 
sympathetic  to  others 

30 

73 

51 

Word  hard,  do  your  best 

10 

30 

54 

Independent  and  self- 
confident 

7 

27 

56 

Know  and  be  true  to 
yourself 

17 

40 

HOWARD 

10 

Be  loyal  and  serve 

71 

Keep  place  neat  and 

country 

20 

3 

clean 

0 

21 

16 

Black  Movement  loyalty 

40 

12 

26 

Polite,  respectful  and 
modest 

13 

0 

44 

Have  sexual  relations 

27 

0 

MANILA 

11 

Follow  customs  and  obey 

26 

Polite,  respectful  and 

laws 

44 

21 

modest 

19 

44 

31 

Help  the  needy 

26 

8 

51 

Work  hard,  do  best 
(males  below  all  other 
college  samples) 

4 

21 

KENYA 

5 

Graduation  from  school 

30 

0 

7 

Be  educated,  have  a  good 
educational  system 

50 

90 

10 

Be  loyal  and  serve 
country 

28 

5 

36 

Honest,  loyal,  sincere, 
responsible 

18 

40 

11 

Follow  customs  and  obey 
laws 

25 

5 

38 

Be  cheerful 

0 

15 

63 

Participate  in  sports 

45 

5 

73 

Attractive  or  correct 
dress  or  grooming 

8 

35 

70 

Be  healthy,  promote 
heal th 

30 

5 

80 

Be  religious 

40 

75 

THAILAND 

23 

Love  and  care  for 

21 

Help  parents 

4 

31 

family 

13 

0 

30 

Help  others 

38 

72 

* 

In 

some  Instances,  readily  visible  by 

inspection , 

the  value  may  be  high  for 

one  sex. 

but 

the  value  for  the  other  sex  is,  nonetheless,  significantly  higher. 


TABLE  9 


BAD  TO  DO" 

'  WITHIN- 

■GROUP  COMPARISON 

OF  MALES 

AND 

FEMALES 

MALES  HIGHER 

FEMALES  HIGHER 

Category 

Idea 

Percentage 

M  F  Category 

Idea 

Percentage 

M  F 

UCLA 


55 

Be  careless  or  reckless 

16 

4 

40 

Live  alone,  no  friends 

0 

12 

77 

Drink  or  get  drunk 

28 

6 

84 

Rape  someone 

8 

0 

NORTHWESTERN 

15 

Conform  to  bad  customs 
and  beliefs  of 

9 

Cheat 

13 

40 

Establishment 

37 

10 

38 

Insincerity  or  hypocrisy 

3 

23 

34 

Hate  or  hold  grudge 

13 

0 

41 

Gossip  or  criticize 
others  behind  their  back 

3 

23 

44 

Have  illicit  or  debasing 
sexual  relations 

20 

0 

73 

Disregard  one's  clothing 
or  personal  appearance 

13 

0 

83 

.  Injure  someone 

20 

3 

HOWARD 

15 

Conform  to  bad  customs 

57 

Not  act  according  to 

and  beliefs  of 

one's  real  feelings  or 

Establishment 

27 

3 

best  convictions 

0 

24 

18 

Betray  one's  country 

13 

0 

60 

Be  the  kind  of  person 
who  cannot  live  life 

26 

Not  get  along  well 
with  other  people 

40 

12 

fully 

0 

27 

82 

Hurt  people's  feelings 
or  cause  trouble  for 
them 

0 

41 

MANILA 

19 

Break  the  law 

33 

13 

20 

Not  obey  or  respect  or 
honor  parents 

33 

87 

28 

Mistreat  one's  boy  or 
girl  friend 

19 

0 

41 

Gossip  or  criticize 
others  behind  their 

46 

Commit  adultery 

26 

8 

back 

7 

31 

76 

Smoke 

19 

0 

54 

Lie  or  be  dishonest 

22 

56 

77 

Drink  or  get  drunk 

30 

3 

74 

Waste  money  or  run  up 
debts 

0 

18 

84 

Rape  someone 

11 

0 

KENYA 

29 

Marry  prematurely  or 

51 

Not  try  hard  to  get 

inappropriately 

38 

5 

ahead 

20 

60 

56 

Be  foolish  or  self- 
deceptive 

5 

25 

82 

Hurt  people's  feelings 
and  cause  trouble  for 
them 

25 

65 

83 

Injure  someone 

18 

55 

THAILAND 

19 

Break  the  law 

13 

0 

26 

Not  get  along  well  with 
other  people 

17 

44 

55 

Be  careless  or  reckless 

38 

9 

73 

Disregard  one's  clothing 

74 

Waste  money  or  run  up 
debts 

50 

9 

or  appearance 

4 

31 

76 

Smoke 

13 

0 

77 

Drink  or  get  drunk 

42 

6 

88 


TABLE  10 

INTERCOLLEGE  VALUE  CORRELATIONS 


Stanford 

UCLA 

North- 

Western 

Howard 

Manila 

Kenya 

Thailand 

USC- 

Thai 

Somchai- 

Thai 

Stanford 

- 

.69 

.67 

.70 

.28 

.18 

,44 

.46 

.30 

UCLA 

.63 

.78 

.79 

.40 

.07 

.37 

.41 

.20 

U 

Northwestern 

.66 

.62 

- 

.74  ' 

.45 

.09 

.62 

.60 

.30 

Howard 

.61 

.73 

.72 

- 

.46 

.14 

.42 

.42 

.13 

Manila 

.41 

.35 

.41 

.63 

- 

.36 

.44 

.53 

.46 

Kenya 

Thailand 

.37 

.46 

.25 

.30 

.49 

.41 

.54 

,33 

.62 

.48 

-  ^ 
.29 

.06 

.13 

© 

.24 

.39 

USC-Thai 

.42 

.19 

.32 

.14 

.24 

.12 

© 

" 

.63 

Somchai-Thai 

.45 

.28 

.43 

.38 

.41 

.27 

.53 

.51 

- 

89 


c,  Intercultural  Value  Comparisons 

The  US  colleges  were  grouped  together  and  compared  as  a  unit  with 
each  of  the  three  foreign  samples,  (The  0,05  level  was  again  used  to 
select  categories  that  differed  significantly.) 


Tables  11,  12,  and  13  show  Kenya,  Thailand,  and  Manila  compared 
with  the  US  goods.  Reading  each  side  of  the  tables  gives  a  picture  of 
some  of  the  ways  in  which  the  United  States  differs  from  other  societies 
and  some  of  the  important  values  in  the  three  foreign  countries.  For 
example,  one  sees  a  US  college  factor  of  independence,  ambition,  a  desire 
to  live  life  fully,  be  true  to  self,  and  love  someone.  The  foreign 
students  value  their  parents  and  elders;  they  are  concerned  with  modesty 
and  politeness. 

If  the  reader  is  knowledgeable  about  any  of  these  three  countries, 
he  will  recognize  that  the  lists  describe  well  some  differences  between 
the  United  States  and  the  particular  country.  Further,  although  the 
samples  were  of  college  students  only,  and  quite  small,  these  category 
titles  describe  values  that  are  important  in  the  whole  culture.  The  lists 
were  checked  with  natives  of  the  three  countries  or  with  experts,  who 
judged  the  information  accurate. 

When  the  three  foreign  countries  are  compared,  lists  like  those  in 
Tables  14  and  15  can  be  generated  by  selecting  the  significantly  different 
items.  Table  14  describes  values  and  behaviors  in  Thailand;  Table  15 
does  the  same  for  Kenya.  The  lists  are  displayed  simply  to  show  the  kind 
of  information  that  can  be  obtained.  The  process  of  selecting  items  is 
mechanical,  but  the  interpretation  requires  a  small  amount  of  knowledge 
for  some  of  the  items.  Thus  the  category  "Go  to  graduate  school"  was 
very  small  in  Thailand  compared  to  the  United  States.  One  must  ask 
whether  it  is  small  because  the  Thai  take  graduate  work  for  granted,  or 
because  they  do  not  expect  to  be  graduate  students  at  all.  Knowing  that 
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TABLE  11 


SIGNIFICANTLY  DIFFERENT  VALUE  CATEGORIES:  KENYA  COMPARED  WITH  US 

Kenya  Higher  US  Higher 


Be  educated  and  have  a  good  educa- 
cational  system 

Be  loyal  to  and  serve  our  country 

Do  something  in  agriculture 

Respect,  obey,  love  and  honor 
our  parents 

Respect  and  obey  elders,  teachers 
and  superiors 

Love  and  forgive  one  another 
Get  married 

Get  a  job  or  have  a  career 

Participate  in  sports  or  athletic 
activities  (males  only) 

Be  religious 

Make  this  a  better  country 


Study  hard  and  get  good  grades 

Protest  against  the  ills  of 
society 

Help  others 

Respect  the  rights  and  dignity  of 
others 

Be  kind,  considerate,  sympathetic 
and  responsive  to  others 

Be  friendly  and  have  friends 

Love  someone  and  be  loved 

Achieve  success  and  recognition 

Be  independent  and  self-confident 

Be  mature  and  self-controlled 

Know  and  be  true  to  yourself 

Be  the  kind  of  person  who  can  live 
life  fully 

Do  things  that  are  fun 
Enjoy  quiet  recreation 
Make  and  save  money 


91 


TABLE  12 


SIGNIFICANTLY  DIFFERENT  VALUE  CATEGORIES:  THAILAND  COMPARED  WITH  US 


Thailand  Higher 


US  Higher 


Help  our  parents  (females  only) 

Be  polite,  respectful  and  modest 

Be  grateful  or  appreciative 

Help  others  (females  only) 

Be  generous  and  unselfish 

Do  things  that  are  fun 

Enjoy  quiet  recreation 

Participate  in  sports  or  athletic 
activities 

Practice  creative  or  expressive 
arts 

Be  moral  and  good,  avoiding 
vices  and  bad  company 


Graduate  from  school  or  college 

Engage  in  extracurricular  activities 

Protest  against  the  ills  of  society 

Be  kind,  considerate,  sympathetic 
and  responsive  to  others 

Have  good  times  with  opposite  sex 

Love  someone  and  be  loved 

Have  sexual  relations 

Get  married 

Achieve  success  and  recognition 

Be  mature  and  self-controlled 

Know  and  be  true  to  yourself 

Be  kind  of  person  who  can  live  life 
fully 

Be  religious 
Make  a  better  world 
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TABLE  13 


SIGNIFICANTLY  DIFFERENT  VALUE  CATEGORIES:  MANILA  COMPARED  WITH  US 


Manila  Higher 


US  Higher 


Be  loyal  to  and  serve  our  country 

Follow  customs  and  obey  the  laws 
of  our  society 

Respect,  obey,  love,  and  honor 
our  parents 

Help  our  parents 

Respect  and  obey  elders,  teachers, 
and  superiors 

Be  polite,  respectful,  and  modest 
Help  others 
Be  religious 

Be  moral  and  good,  avoiding  vices 
and  bad  company 


Graduate  from  school  or  college 

Get  and  use  information 

Love  and  take  good  care  of  family 
and  spouse 

Be  kind,  considerate,  sympathetic  and 
responsive  to  others 

Have  good  times  with  the  opposite  sex 
Love  someone  and  be  loved 
Have  sexual  relations 
Get  married 

Work  hard,  do  your  best,  organize 
your  time  well 

Achieve  success  and  recognition 

Know  and  be  true  to  yourself 

Be  the  kind  of  person  who  can  live 
life  fully 

Do  things  that  are  fun 

Enjoy  quiet  recreation 

Participate  in  sports  or  athletic 
activities 

Practice  creative  or  expressive  arts 
Take  a  trip 
Make  and  save  money 
Drive  carefully 
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TABLE  14 


THAILAND:  STATEMENTS  PREPARED  FROM  ANALYSIS  OF  COMPARATIVE  DATA 


Polygamy  exists  in  the  Philippines,  Kenya,  and  to  a  lesser  extent  in 
Thailand, 

The  Thai  gamble  more  than  the  others. 

The  Thai  do  not  see  religion  as  a  separate  institution,  apparently  because 
it  is  so  integrated  into  their  lives. 

Thai  college  students  expect  to  graduate  from  college,  but  do  not  expect 
to  go  further. 

Customs  and  traditions  are  extremely  important  to  the  Thai. 

Getting  along  with  people  is  more  important  to  the  Thai  than  to  people 
from  the  Philippines  or  Kenya. 

Thai  are  very  conscious  of  status  differences  and  disapprove  of  the  way 
some  people  handle  superior  status. 

Thai  gossip  a  lot. 

Premarital  relations  are  not  as  common  in  Thailand  as  in  the  United  States, 
but  Thai  who  visit  the  United  States  learn  about  it. 

Thailand  has  many  accidents . 

Thai  play  a  lot,  but  occasionally  feel  guilty  about  it. 

In  Thailand,  hard  work  is  not  considered  a  virtue. 

Thai  men  are  careless  with  money;  they  like  to  gamble. 

It  is  very  important  to  the  Thai  to  do  the  morally  correct  thing. 

Thai  view  property  (things,  not  land)  differently  than  others,  but  it  is 
not  clear  how.  (Possibility:  theft  is  so  common  it  is  not  salient. 

Maybe  the  Thai  expect  to  be  tricked  and  have  things  stolen.) 

Thai  believe  that  some  of  their  government  officials  are  corrupt,  and  they 
care  about  it. 

Drinking  is  a  problem  among  Thai  men. 
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TABLE  15 


KENYA:  STATEMENTS  PREPARED  FROM  ANALYSIS  OF  COMPARATIVE  DATA 


The  land  and  agriculture  is  important  to  Kenya  in  a  way  that  is  quite 
different  from  any  other. 


Smuggling  is  important  in  Manila  and  Kenya;  not  in  others. 


Polygamy  exists  in  the  Philippines,  Kenya,  and  to  a  lesser  extent  in 
Thailand. 


Manila  and  Kenya  do  not  use  illegal  or  crippling  drugs. 

Violence  is  not  uncommon  in  Kenya. 

Prostitution  is  more  visible  in  Kenya. 

Drinking  is  either  a  major  problem  in  Kenya  or  it  occurs  infrequently. 

Getting  along  with  people  is  more  important  to  the  Thai  than  to  people 
from  the  Philippines  or  Kenya. 

Religion  and  religious  institutions  are  important  in  the  lives  of  the 
people. 

Marrying  properly  is  very  important  in  Kenya. 


Kenya  citizens  expect  their  government  to  do  many  things,  but  they  believe 
the  government  is  inept. 


Thailand  has  comparatively  few  universities,  one  may  conclude  that  not 
many  Thai  expect  to  go  to  graduate  school. 

C.  THAI  STUDY 

1.  Translation  and  Forms  of  the  ECHO  Questions 

Early  in  the  project,  Thai  experts  were  asked  to  anticipate  problems 
that  might  arise  in  a  study  in  Thailand.  One  of  the  first  problems  was 
whether  the  standard  ECHO  questions  could  be  translated  into  Thai  and 
carry  the  same  meanings  as  in  English.  The  experts  independently  agreed 
on  the  translation  for  the  key  word  good:  di. 

Almost  a  year  later,  when  the  ECHO  staff  members  were  actually 
preparing  the  Thai  question  cards,  they  recognized  that  although  the  word 
di  carries  some  of  the  meanings  of  good,  the  flavor  is  of  moral  good,  not 
good  in  the  sense  of  pleasure.  To  make  the  Thai  data  comparable  with 
data  from  other  countries,  two  forms  of  the  question  were  prepared: 

Form  L,  ”What  is  something  you  like  [cho.b]  to  do?”  and  Form  0,  ”What  is 
something  you  ought  [di]  to  do?”  Form  D,  ”Not  like”  and  Form  S,  ”ought 
not”  were  used  as  disvalues .  (The  letters  D  and  ^  may  be  remembered  as 
the  initials  of  Dislike  and  Shun.)  Figures  3  and  4  show  the  four 
question  cards. 

Thai  S_s  apparently  experienced  no  difficulty  with  the  new  forms  of 

the  question.  The  E  concluded  that  the  four  Thai  questions  and  the  two 

English  questions  yield  answers  that  are  in  general  equivalent,  but  the 

Thai  good  questions  are  handled  differently  from  the  bad  ones.  Some 

categories  in  the  Thai  goods  were  made  up  largely  of  like-to  cards  (for 

example,  ”Do  things  for  enjoyment,”  "Watch  entertaining  things")  and 

some  largely  of  ought-to  cards  (such  as,  "Be  a  good  relative,"  "Follow 

the  rules”).  The  two  Thai  bads  did  not  separate  in  this  way. 

*  1  . 

The  LCDS  question  set  was  first  tested  on  five  Thai  who  were  studying 

at  the  University  of  California,  Los  Angeles 
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^4  n  niwVe>t)  ni _ 

Sli]  thii  than  choob  tham 


C|  /  y  I  >'  I  . 

I  ri  t  -U  f/ail  0  «J /1 1/ ^  ^  ^ 


V 


Khraj  thll  hen  chSob  duaj  kab  than  thia  than  tham 
_ _ _ _ _ _ Prood  toob  khamthaam  naa  too  paj  * 


What  do  you  like  to  do? 

Who  would  approve  if  you  did  it? 


liiVeiini _ 

Sii]  thii  than  maj  choob  tham 


^  j  <  t/  I 

i^ir^  miwux^ti^nun^tnu/nniuYii _ 

Khraj  thll  maj  hen  choob  duaj  kab  than  thaa  than  tham 


14^ 


^el  0O<^i/iijiviui(n 


T^iJ 


What  do  you  not  like  to  do? 

Who  would  disapprove  if  you  did  it? 


Figure  3.  The  L  and  D  Cards  for  Thailand  Study 


Slri  thil  than  khid  wa  pen  khwaamdii  thil  khuan  kratham 


q  ^  ^  I  </  •  ' 

- - - - - 

Khraj  thli  hen  choob  duaj  kab  than  thaa  than  tham  ^ 

« i)  in  1  }J  v\u J 

Prood  toob  khamthaam  naa  too  paj 


What  do  you  think  is  a  good  thing  which  one  ought  to  do? 
Who  would  approve  if  you  did  it? 


^  '  — >s  ‘  H  ^ 

Slr|  thil  than  khld  waa  maj  khuan  kratham 


- - - — 

Khraj  thil  mai  hen  ch^jb  duaj  kkb  than  thaa  than  tham. 


What  do  you  think  one  ought  not  to  do? 
Who  would  not  approve  if  you  did  it? 


Figure  4. 


The  0  and  S  Cards  for  Thailand  Study 


2. 


Subjects 

Professor  Ravipan  Somnapan  of  Bangkok  became  interested  in  the 
potential  of  the  ECHO  method  and  arranged  informally  with  Barthol  to  use 
ECHO  in  two  Thai  classes  at  the  Chulalongkorn  University  and  the 
College  of  Education  (Prasarnmit) .  A  graduate  class  in  educational 
computer  programming  (N  =  26)  and  an  undergraduate  class  is  psychology 
(N  =  15)  composed  the  sample.  There  were  19  male  _Ss  and  22  female  ^s . 

Ages  ranged  from  20  to  38.  All  read  English,  and  most  spoke  it  with 
reasonable  fluency. 

As  part  of  their  training  and  to  conduct  a  pilot  project  in  the 
Thai  language,  ECHO  staff  members  David  and  Chintana  Haas,  under  Bridge’s 
direction,  recruited  and  surveyed  23  Thai  students  attending  the 
University  of  Southern  California.  The  resulting  data  are  used  for 
comparison  with  the  Ravipan  samples. 

Later,  in  Bangkok,  the  Haases  also  collected  data  from  an  available 
group  of  11  male  Thai  field  interviewers  and  their  director.  Dr.  Somchai 
Rakivichit.  All  ^s  were  college  graduates.  These  data  are  also  used 
for  comparison  with  the  Ravipan  samples. 

3.  Data  Collection 

Two  sets  of  test  packets  had  been  prepared:  LD  (Like-Dislike;  good 
and  bad  things  one  would/would  not  like  to  do)  and  OS  (Ought-Shun;  good 
and  bad  things  one  ought/ought  not  to  do) .  The  packets  were  randomly 
passed  out  so  that  one-half  of  the  computer  class  received  LD  packets, 
the  other  half  OS  packets;  each  ^  in  the  psychology  class  received  both 
packets,"  one  half  did  LD  first,  the  other  half  OS  first. 

Each  of  the  15  _S^s  in  the  psychology  class  was  counted  as  one  person 
answering  LD  questions  and  another  person  answering  OS  questions.  When  L 
and  0  or  D  and  S  questions  were  combined  in  this  study  and  in  the  Inter- 
cultural  College  Study,  computer  outputs  showed  30  ^s  in  the  psychology 
class,  or  56  ^s  in  the  Thailand  sample  (computer  and  psychology  class 
together).  To  test  whether  this  overweighting  of  the  psychology  class 
made  a  difference,  the  data  were  reanalyzed  counting  each  ^  only  once,  and 
results  before  and  after  the  corrections  were  correlated.  The  correlation 
was  +0.98,  indicating  that  the  effect  of  overweighting  was  negligible. 
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The  E  gave  standard  ECHO  instructions  in  English  after  an  intro¬ 
duction  by  Dr.  Ravipan.  The  time  for  completion  was  approximately 
one-half  hour.  Each  _S  wrote  his  age  and  sex  on  the  envelope. 

4.  Classifications 

The  data  were  classified  in  various  ways,  for  various  purposes. 

1.  As  instruction  for  Dr.  Ravipan ’s  psychology  students,  the 
class  was  divided  into  four  teams  of  four  persons  (a  returning 
student  made  the  sixteenth  member;  he  was  not  an  S_) ,  each 

team  classifying  one-half  of  either  goods  or  bads  (LO  together, 
DS  together).  On  completion,  the  two  teams  working  with  goods 
merged  to  consolidate  the  two  classifications  of  the  bads;  the 
other  two  teams  merged  and  consolidated  the  goods.  This 
classification  session,  then,  consisted  of  one  team  of  eight 
classifying  bads.  The  E  observed  that  eight  classifiers 
working  as  a  unit  do  not  make  an  efficient  team. 

2.  Six  members  of  the  computer  class  volunteered  to  classify 
their  data  in  teams  of  three  . 

3.  D.  and  C.  Haas  classified  all  of  the  Thai  data  together:  the 
USC-Thai  and  the  Somchai-Thai  research  team  data,  as  well  as 
the  computer  class  and  psychology  class  data. 

4.  The  responses  from  all  four  groups  were  translated  by  the 
Haases,  and  reclassification  decks  (see  Glossary,  Appendix  I) 
were  prepared.  These  became  part  of  the  Intercultural 
College  study  and  were  classified  with  the  other  college  data. 

5.  Effects  of  the  LOPS  Question  Set 

After  classification  of  the  Thailand  and  USC-Thai  data,  the  Like  (L) 
and  Ought  (0)  cards  were  separated  to  see  whether  the  results  would 
change;  in  the  USC-Thai  sample,  the  D  and  S  cards  were  separated  also. 

^Pacific  Technical  Analysts,  Inc.,  Bangkok,  graciously  lent  keypunch 
equipment  for  the  first  two  classifications. 


100 


The  USC-Thai  categories  proved  virtually  identical  after  the 
separation.  Only  one  category  in  goods  or  bads  had  a  discrepancy  that 
approached  statistical  significance  ("To  make  good  use  of  one's  free 
time,"  12  L  cards  vs  five  0  cards).  In  the  bads,  a  few  minor  changes 
in  ranking  would  have  occurred  if  D  or  S  cards  had  been  used  alone; 
one  small  category  would  have  disappeared  if  S  had  not  been  used,  two  if 
D  had  not  been  used. 

Table  16  shows  the  distribution  of  L  and  0  cards  for  the  psychology 
class;  significant  differences  occur  in  six  of  the  20  categories. 

Table  17  shows  those  categories  in  the  psychology  and  computer 
classes  that  had  significantly  more  L  or  0  cards.  Relaxation,  fun, 
hobbies,  entertainment,  enjoyment,  and  travel  are  things  one  likes; 
honesty,  respect,  politeness,  and  self-confidence  are  obligations. 

From  this  brief  analysis,  it  appears  that  the  LODS  question  set 
can  discriminate  usefully  between  preferences  and  obligations. 

6.  US  Classification  of  Thai  Data 

When  classifiers  are  not  indigenous,  they  can  misunderstand  the 
meanings  of  responses  that  would  have  been  quite  clear  to  an  indigen. 

A  team  of  three  US  college  students  classified  the  Thai  samples  (along 
with  the  other  college  data) .  Classification  of  each  Thai  sample  was 
checked  against  an  earlier  indigenous  classification.  Some  responses, 
though  rationally  classified,  had  not  been  interpreted  by  the  US  team  as 
the  Thai  would  have  interpreted  them.  The  US  team  had  placed  responses 
about  cooking,  planting  trees,  and  other  apparently  domestic  things 
under  "Be  kind  to  parents";  the  Thai  had  classified  them  under 
"Relaxation."  Table  18  shows  the  different  ways  the  two  teams  viewed 
seven  sets  of  responses;  each  pair  of  numbered  titles  was  applied  to  the 
same  set  of  responses. 

T.W.  Milburn  has  collected  equivalent  data  in  English  from  34  DePaul 
University  students  to  evaluate  the  LODS  set,  but  the  analysis  is  not 
complete  at  this  writing. 
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TABLE  16 


PSYCHOLOGY  CLASS:  LIKE  VS  OUGHT 


Category 


No. 

Like 

Ought 

7 

To  seek  education 

12 

9 

15 

Relax  and  go  places  for  fun 

13 

2' 

1 

Help  others 

9 

12 

10 

Follow  principles  of  religion 
and  customs 

7 

10 

19 

Have  hobbies 

10 

2" 

5 

Respect  others  rights 

4 

9 

3 

To  be  honest 

3 

* 

9 

8 

Work  hard 

4 

8 

6 

Have  self-confidence 

1 

7“ 

13 

Take  exercise  and  engage  in 
sports 

7 

5 

18 

Make  others  happy 

5 

3 

2 

Do  things  useful  to  society 

3 

7 

16 

Know  how  to  get  along  in 
society 

7 

2 

12 

Respect  older  people 

1 

* 

7 

17 

Do  things  which  make  one 
feel  happy 

4 

4 

9 

Listen  to  others 

4 

3 

14 

Be  neat  and  clean 

3 

3 

20 

Miscellaneous 

3 

3 

11 

Improve  oneself 

1 

3 

4 

Repay  favors 

0 

4" 

* 

Categories  in  which  the  number  of  Like  (L)  cards 
differs  significantly  from  the  number  of  Ought  (0) 
cards  (p  <  0.05). 


TABLE  17 


COMPARISON  OF  LIKE  AND  OUGHT 


Category 

No. 


No.  of  Cards 
Like  Ought 


15 

19 


18 

6 

12 

4 


Psychology  Class 


13 
10 

Categories  in  which  Ought  is 
significantly  greater  than  Like: 


To  be  honest  3 
Have  self-confidence  1 
Respect  older  people  1 
Repay  favors  0 


Categories  in  which  Like  is 
significantly  greater  than  Ought: 

Relax  and  go  places  for  fun 

Have  hobbies 


2 

2 


9 

7 

7 

4 


Computer  Class 


Categories  in  which  Like  is 
significantly  greater  than  Ought: 

10 

To  watch  entertaining  things 

8 

0 

12 

To  do  things  for  enjoyment 

12 

1 

14 

To  travel 

8 

1 

Categories  in  which  Ought  is 
significantly  greater  than  Like: 

1 

To  treat  others  with  respect, 
politeness  and  honesty 

1 

11 

3 

To  be  a  good  relative 

2 

8 

7 

To  support  the  nation 

2 

7 
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TABLE  18 


DISCREPANCIES  BETWEEN  US  AND  THAI  CLASSIFICATIONS  OF  THAI  RESPONSES 


US  Category  Title  Thai  Category  Title 


1. 

Practice  creative  or 
expressive  arts 

1. 

Do  things  for  relaxation 

2. 

Participate  in  sports 

2. 

Do  things  for  relaxation 

3. 

Be  kind  to  parents  (these 
responses  were  about 
domestic  chores,  such  as 
to  cook) 

3. 

Do  things  for  relaxation 

4. 

Not  follow  customs 

4. 

Lack  neatness  and  politeness 

5. 

Be  dishonest  (these 
responses  were  about 
bribery) 

5. 

Be  unpatriotic 

6 . 

Gamb le 

6  • 

Not  be  responsible 

7. 

Steal 

7. 

Be  immoral 

7 •  Thai  Sources  of  Approval  and  Disapproval 

ECHO  sources  of  approval  and  disapproval  provide  useful  information 
in  identifying  the  authority  figures  in  a  culture.  The  Thai  data  indicate 
that  parents  are  important  authorities  for  both  males  and  females,  but 
the  father  is  more  important  to  daughters  than  to  sons;  47  percent  of  the 
females  and  17  percent  of  the  males  named  Father  as  a  source  of  approval; 
75  percent  of  the  females  and  21  percent  of  the  males  named  Father  as  a 
source  of  disapproval.  (The  Kenya  females  responded  similarly.)  The 
father-daughter  relationship  in  Thailand  seems  to  be  quite  different 
from  that  in  the  United  States. 

The  use  of  Self  as  a  source  may  indicate  the  internalization  of 
values  and  a  willingness  to  be  responsible  for  one’s  own  actions 
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(Barthol  and  Bridge,  1967).  In  the  Thai  sample,  Self  was  named  by  62 
percent  of  both  males  and  females.  Both  sexes  named  Self  with  equal 
frequency  in  goods  and  bads,  but  Self  appeared  significantly  more  often 
on  L  (Like)  and  D  (Dislike)  than  on  0  (Ought)  and  S  (Ought  Not)  cards, 
indicating  a  greater  correspondence  between  preference  and  self-approval 
than  between  obligation  and  self-approval.  Not  one  female  cited  Self  on 
an  0  card;  only  two  did  so  on  S  cards;  the  difference  here  between  males 
and  females  is  significant  at  the  0.05  level.  Although  both  males  and 
females  look  to  outside  sources  for  praise  and  punishment  for  behaviors 
that  fall  in  the  moral  good  or  bad  categories,  the  females  look  outside 
almost  exclusively. 

Additional  substantive  findings  about  Thailand  are  reported  in  the 
Intercultural  College  Study,  in  Tables  8,  9,  12  and  14. 

D .  KENYA  STUDY 

1 .  Subjects 

ECHO  data  were  collected  in  the  standard  way  from  English-speaking 
black  students  in  Kenya,  for  inclusion  in  the  Intercultural  College 
Study.  The  sample  comprised  40  male  and  20  female  students,  age  16  to 
35.  Some  were  psychology  students  in  a  teacher’s  college;  others  were 
high  school  students.  These  educational  levels  were  held  to  be  rough 
Kenyan  equivalents  for  the  college  samples  in  the  other  cultures. 

2 .  Data  Classification 

The  data  were  classified  in  Kenya  by  indigens.  Then  the  cards 
were  returned  to  the  United  States,  where  they  were  processed  and 
classified  again  by  the  college  classification  team. 

•k 

T.W.  Milburn  and  Marc  Ross,  the  latter  a  political  scientist  at  Bryn 
Mawr  College  and  an  expert  on  Kenya,  made  arrangements  with  their 
colleague,  E.M.B.H.  Ombogodonga,  to  collect  the  data  and  administer  the 
indigenous  classifications. 
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3. 


Characterizing  the  Culture  from  Classified  Responses 
After  use  of  the  Kenya  sample  in  the  College  Study,  a  display  deck 
was  prepared  (see  Glossary,  Appendix  I,  for  definition),  and  all 
responses  were  listed  in  their  College-Study  categories,  Barthol  used 
this  list  to  test  the  proposition  that  individual  ECHO  responses, 
appropriately  classified,  reveal  many  things  about  the  culture  from 
which  they  come.  This  use  of  ECHO  responses  rests  on  an  assumption  that 
respondents,  even  when  facetious,  mention  things  that  do  exist  in  their 
culture  (and  that  they  know  about)  and  do  not  mention  things  that  do  not 
exist  (or  that  they  do  not  know  about)  . 

The  essay  presented  below  is  about  Kenya,  and  was  written  by 
K.  Bucknam,  a  college  student  and  a  member  of  the  College-Study  classi¬ 
fication  team,  working  from  Bartholds  notes  (listed  in  Table  19).  Before 
conducting  this  exercise,  Bucknam  knew  virtually  nothing  about  Kenya, 
and  Barthol  knew  only  that  Kenya,  an  English-speaking  country,  is  on  the 
east  coast  of  Africa,  is  populated  largely  by  blacks,  and  was  a  scene 
a  few  years  ago  of  Mau  Mau  fighting. 

The  description  is  of  interest  primarily  because  of  the  amount  of 
information  obtained  inexpensively  and  quickly.  Whenever  an  expert 
could  write  a  similar  document,  in  more  detail  and  with  greater  assurance, 
this  use  of  ECHO  would  be  redundant.  However,  in  a  particular  village 
or  area  for  which  no  current  expert  is  available,  this  inductive  applica¬ 
tion  of  ECHO  could  be  of  great  value. 

ECHO  Description  of  Kenya 


Kenya  has  a  rugged  coastline  and  an  important  harbor 
facility.  The  normal  shipping  trade  is  supplemented  by  a 
considerable  amount  of  smuggling.  The  large  forest  pre¬ 
serves,  in  which  wildlife  is  protected,  attract  tourists, 
an  industry  which  the  government  is  trying  to  increase. 
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The  vital  rail  network  largely  substitutes  for  the  highway 
system,  which  is  mostly  unpaved;  macadam  rather  than  con¬ 
crete  is  used  for  those  roads  that  are  paved.  The  East 
African  Power  and  Lighting  Company  provides  power,  but 
electricity  is  still  a  luxury  found  only  in  large  cities. 
The  high  mountains  of  Kenya  are  seen  as  challenges  to 
mountaineers.  Most  of  the  urban  homes  have  gardens. 

Agriculture  plays  a  major  role  in  the  nation’s 
economy.  Money  crops  have  only  recently  been  introduced. 
The  outmoded  slash-and-burn  system  of  agriculture  is  still 
used,  but  the  government,  in  the  person  of  the  Minister  of 
Agriculture,  is  trying  to  introduce  new  methods.  Insects 
cause  considerable  damage. 

The  government  of  Kenya  is  paternalistic,  and  the 
people  expect  it  to  be  that  way;  even  the  multiparty 
system  has  not  changed  this  attitude.  The  new  govern¬ 
ment  is  trying  to  cope  with  the  problem  of  modernization. 

The  numerous  tribes  of  the  country  are  jealous  and 
quarrelsome.  Trying  to  unify  Kenya,  the  government  is 
backing  an  attempt  to  make  Swahili  the  official  language. 

Kenya  currently  enjoys  a  strong  relationship  with 
India.  Harambee  is  strong,  but  not  currently  in  power. 

The  Maragori  have  low  status  in  the  country. 

Universal  education  has  not  yet  arrived  in  Kenya, 
though  the  current  Minister  of  Education  is  powerful 
and  is  trying  to  upgrade  the  school  system.  Keeping 
children  in  school  is  a  problem. 


The  people  of  Kenya  are  quite  generous,  perhaps  because 
of  their  religious  training.  Among  many  Christians,  God  is 
seen  as  a  personal  powerful  figure.  Christianity,  to  a  large 
extent  non-Catholic ,  is  increasing  its  hold,  and  currently 
vies  with  pagan  tribal  religions  for  acceptance  by  the 
populace . 

Having  many  children  is  valued,  although  the  problem  of 
overpopulation  is  somewhat  changing  that  concept.  Children 
are  actively  taught  the  virtues  and  sins,  the  importance  of 
obedience  to  parents,  and  respect  for  both  parents  and 
elders.  The  father  is  a  significant  person  in  the  family. 

Polygamy  and  intrafamily  marriages  are  not  uncommon. 
Premarital  intercourse  leads  to  marriage  (presumably  if  the 
girl  is  of  acceptable  tribe  or  status).  Incest  and  adultery 
are  disapproved  but  exist.  The  custom  of  paying  dowries  for 
wives  is  a  common  practice,  even  though  to  do  so  may  leave 
the  male^s  family  impoverished. 

The  young  men  of  Kenya  travel  in  groups  and  encourage 
each  other  to  do  "bad"  things.  They  often  drink  and  con- 
sort  with  prostitutes,  who  are  fairly  numerous.  Sports  and 
athletics  are  very  popular,  particularly  in  the  schools. 

The  status  of  women  is  high.  Education  is  considered 
important  for  women.  Though  not  many  are  educated,  those 
women  who  are,  have  made  valuable  contribution  to  the 
sciences  and  professions.  More  women  are  moving  into 
medicine.  The  girls  are  encouraged  by  family,  church,  and 
school  to  be  industrious. 


Violence  is  common.  People  beat  those  close  to  them: 
children  beat  their  parents,  students  beat  their  teachers, 
and  even  girls  sometimes  resort  to  violence  and  fighting. 

Killing  is  a  real  problem,  and  theft  and  robbery  are  wide¬ 
spread.  Reckless  driving,  particularly  on  the  poor  and 
narrow  roads,  is  a  major  problem.  Drugs  are  not  widely  used. 

This  description  of  Kenya  was  checked  sentence  by  sentence  with 
Marc  Ross.  Most  of  the  statements  were  judged  accurate.  Corrections 
were : 

1.  Most  rural,  not  urban,  homes  have  gardens, 

2.  The  Kenya  government,  although  paternalistic,  is  not  strong 
and  the  opposition  is  not  effective. 

3.  Harambee  is  a  phrase  that  means  approximately  "upward  and 
onward. " 

4.  Keeping  children  in  school  is  not  a  problem. 

5.  The  status  of  women  is  beginning  to  rise;  women  have  entered 
the  professions  but  apparently  have  not  yet  made  major 
contributions . 

6.  It  is  not  clear  whether  the  beating  should  be  interpreted 
literally  or  symbolically. 

7.  Maragori  should  be  spelled  Maragoli;  the  mistake  is  common 
in  Kenya. 

The  Encyclopedia  Brittanica,  1962  edition,  was  used  to  check 
accuracy  further  and  to  determine  whether  the  responses  yielded  any  infor¬ 
mation  not  included  in  the  reference  work.  Everything  in  the  essay  that 
was  also  mentioned  by  the  encyclopedia  was  confirmed.  Some  of  the  essay 
items  not  mentioned  in  the  encyclopedia  article  are  as  follows: 
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TABLE  19 


UNEDITED  LIST  OF  KENYA  IMPRESSIONS  FROM  ECHO  RESPONSES 


Language:  Swahili.  But  not  spoken  by  everyone. 
Probably  many  people  speak  English;  push  by 
government  to  make  Swahili  official. 

Kenya  has  a  new  government. 

The  Minister  of  Education  is  a  powerful  man. 

Kenya  is  trying  to  upgrade  education. 

Women  are  important  and  are  seen  as  important 
contributors  to  science  and  the  professions. 

Government  is  paternalistic  and  the  people  expect 
it  to  be  that  way. 

Polygamy  is  accepted;  having  many  children  is 
valued,  but  overpopulation  is  a  recognized 
problem. 

The  people  are  generous. 

The  Minister  of  Agriculture  is  a  good  propagandist. 
It  is  important  to  respect  parents  and  elders. 

God  is  important. 

Education  for  women  is  important,  but  not  many  are 
yet  educated. 

The  East  African  Power  and  Lighting  Company  is  in 
Kenya. 

Organized  sports  is  important  at  school. 

Some  of  Kenya  is  not  well  explored;  it  has  high 
mountains . 

Kenya  has  National  Parks  that  tourists  visit. 

These  are  used  as  wild  life  preserves . 

Insects  are  a  problem. 

Christianity  is  Important  in  Kenya.  Probably  not 
a  Catholic  country. 

Many  tribes  exist.  They  are  cohesive;  there  is 
friction. 

Having  money  crops  is  recent. 

At  least  two  political  parties  exist. 

India  has  some  strong  relationship  with  Kenya. 

Are  Indians  part  of  the  merchant  class? 

Many  see  birth  rate  as  too  high. 

The  dowry  is  common  practice,  but  it  sometimes 
hurts  very  much.  Men  pay  dowries  for  their  wives. 

People  are  ambivalent  about  the  current  president. 

Electricity  is  not  widespread. 

Kenya  may  not  have  a  good  harbor,  but  it  is 
possible.  It  has  a  fairly  long  coastline, 
probably  rugged. 


Many  roads  are  not  paved.  Concrete  is  not  widely  used. 

They  have  a  railroad  system.  Probably  very  important 
to  economy. 

Keeping  children  in  school  is  a  problem. 

The  Harambee  (a  tribe?  probably  a  political  party)  are 
strong  but  are  not  in  power. 

The  slash  and  bum  system  of  agriculture  is  still  used, 
but  the  government  is  trying  to  stop  it. 

Obedience  to  parents  is  taught  vigorously. 

Violence  is  common.  People  beat  those  close  to  them. 

Teachers  occasionally  get  beaten  by  pupils. 

The  Maragori  have  low  status. 

Intra-family  marriages  are  not  uncommon.  Incest  is 
disapproved  but  exists. 

Adultery. 

Premarital  intercourse  leads  to  marriage  (presumably  if 
the  girl  is  of  acceptable  tribe  or  status). 

Children  are  actively  taught  the  virtues  and  sins. 

Automobiles  are  driven  recklessly. 

Prostitution  is  fairly  common. 

Smuggling  is  an  industry. 

Tribal  religions  vie  with  Christianity  for  hold  over 
the  people.  Paganism. 

The  young  men  travel  in  groups  and  encourage  each  other 
to  do  "bad"  things. 

Killing  is  a  real  problem. 

Theft  and  robbery  is  widespread. 

Young  men  drink  and  consort  with  prostitutes. 

Juggery  and  changa  have  some  importance  in  trade,  (I 
do  not  understand  what  this  means) .  The  structure  of 
trade  probably  is  different.  What  are  ’Co.  societies’? 

Gardens  are  Important. 

A  child  sometimes  beats  his  parents. 

Girls  are  trained  to  be  industrious. 

Girls  sometimes  resort  to  violence  and  fighting. 

They  don't  use  drugs  much. 

The  father  is  a  significant  person  in  the  family. 

Women  are  moving  into  medicine. 

Universal  education  has  not  yet  arrived. 
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Smuggling 

Slash- an d-b urn  agriculture 

The  high  value  placed  on  having  many  children 
Polygamy 

Respect  for  elders 
Dowries 

Recreation  practices  of  young  men 
Prostitution 

Women  in  the  professions 
Reckless  driving 
Gambling 
Drinking 

These  are  significant  items.  The  authors  conclude  that  the  described 
inductive  use  of  ECHO  may  well  serve  as  a  check  on  or  supplement  to 
other  sources  of  information  about  unfamiliar  cultures. 

E.  EAST  LOS  ANGELES  SKILL  CENTER  STUDY 


1.  Purpose  of  the  Study 

The  study  was  originated  in  preparation  for  collecting  ECHO  data 
by  oral  interviews  from  Northeast  Thai  peasants,  some  of  whom  cannot 
read. 


a.  Feasibility  of  ECHO  Interviews 

The  first  objective  was  to  test  the  feasibility  of  collecting  ECHO 
data  by  oral  interview  survey  rather  than  written  survey.  Specific 
questions  to  be  answered  were: 

1.  Do  respondents  readily  give  ten  oral  answers  to  the  same 
pair  of  ECHO  questions? 

2.  How  much  time  is  consumed  in  the  interview? 


Ill 


b.  Indigenous  Classification 

The  second  objective  was  to  observe  classification  of  ECHO  data  by 
indigens  having  less  education  than  indigenous  classifiers  in  earlier 
studies.  Specific  questions  were: 

1.  Can  classifiers  having  limited  reading  skills  carry  out  the 
procedures  of  classification? 

2.  How  much  time  is  required? 


3.  What  problems  arise  where  classification  is  supervised  through 
an  intermediary  by  an  ECHO  staff  member  who  does  not  under¬ 
stand  the  language  of  the  subject  group? 


4.  Does  the  indigenous  classification  provide  useful  information? 


c.  Characteristics  of  ECHO  Interview  Data 

A  third  objective  was  to  explore  differences  between  written  and 
interview  ECHO  data.  Though  this  objective  had  to  be  subordinated  to  the 
first  two,  some  progress  could  be  made  toward  posing  specific  questions 
about  differences  arising  from  the  interpersonal  character  of  the  inter¬ 
view,  the  greater  verbal  productivity  of  the  oral  respondent,  and  the 
lack  of  anonymity  in  the  interview. 


2.  Search  for  Population 

Some  difficulty  was  experienced  in  finding  a  suitable  population. 
Two  attempts  were  made  to  assemble  a  Spanish-speaking  group  at  the 
Murphy  Ranch,  in  East  Whittier,  but  descriptions  of  the  population  given 
by  the  administration  proved  unreliable.  A  group  of  negroes  in  Watts 
was  considered  next,  but  arrangements  could  not  be  worked  out.  Finally, 
an  accessible  and  suitable  group  of  Spanish-speaking  immigrants  was 
found  at  the  East  Los  Angeles  Skill  Center,  a  project  of  the  Division  of 
Adult  Education  of  the  Los  Angeles  City  School  Districts. 
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The  East  Los  Angeles  Skill  Center  is  funded  under  the  Manpower 
Development  and  Training  Act  to  increase  employability  through  vocational 
training,  basic  education,  and  English  as  a  second  language.  The 
administration  of  the  Center  was  willing  to  participate  in  the  research, 
and  all  of  the  students  and  personnel  involved  were  most  cooperative  and 
helpful. 

Additional  data  were  sought  from  a  group  of  negroes  at  the  Long 
Beach  Community  Center,  but  the  group  became  inaccessible  when  the 
administration  changed. 

:k 

3.  Selection  of  Samples 

a.  Literacy  Test 

As  an  analog  of  the  illiterate  peasant,  "non-readers"  were  identi¬ 
fied  at  the  Skill  Center  by  asking  students  to  read  a  letter  taken  from 
a  third-grade  Spanish  textbook  and  then  answer  four  multiple-choice 
reading-comprehension  questions  based  on  the  letter.  In  consultation 
with  teachers  at  the  Center,  the  non-reader  was  defined  as  a  student  who 
missed  any  question  on  this  test.  Students  who  missed  no  questions  were 
called  "readers." 

b .  Interviewed  Group 

Fifteen  male  students  were  selected  for  interview.  Ten  of  these 
were  non- readers.  The  other  five  were  readers  who  resembled  the  non¬ 
readers  in  age,  years  spent  in  the  United  States,  level  of  Spanish  educa¬ 
tion,  and  marital  status.  The  statistical  description  of  these  two 
subsamples  is  given  in  the  first  part  of  Table  20. 


W.H.  Hunting  and  E.P.  Hunting  selected  the  samples,  prepared  the 
instructions,  and  supervised  the  data  collection  and  indigenous  classifi¬ 
cation.  J.J.  Sanchez,  of  the  Skill  Center,  acted  as  field  assistant. 
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TABLE  20 


EAST  LOS  ANGELES 

SKILL 

CENTER— DESCRIPTION  OF 

SAMPLE 

Probability  o 

Std. 

Std. 

Difference 

Mean 

Dev , 

%  Mean  Dev.  % 

(Two-Tailed) 

Interviewed 
Non-Reader  Group 
(Male,  N=10) 

Interviewed 

Reader  Group 
(Male,  N=5) 

Age 

34.4  12.6 

31.0 

5.3 

* 

NS 

Years  in  US 

8.6  8.1 

8.0 

6.1 

NS 

Spanish  Education 
Beyond  Primary  Grades 

0.0 

0.0 

NS 

Married 

60.0 

80.0 

NS 

Interviewed  Group 
(Male,  N=15) 

Written  Survey 
(Male,  N=15) 

Age 

33.3  10.9 

33.5 

8.7 

NS 

Years  in  US 

8.4  7.5 

6.5 

6.8 

NS 

Spanish  Education 
Beyond  Primary  Grades 

0.0 

66.7 

0.000 

Married 

66.7 

66.7 

NS 

Written  Survey 
(Male,  N=15) 

Written  Survey 
(Female,  N=15) 

Age 

33.5  8.7 

33.2 

11.4 

NS 

Years  in  US 

6.5  6.8 

6.9 

5.2 

NS 

Spanish  Education 
Beyond  Primary  Grades 

66.7 

73.3 

NS 

Married 

66.7 

6.7 

0.002 

Probability  greater  than  0.05  is  called  Non-Significant  (NS). 
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One  purpose  of  including  readers  as  well  as  non-readers  was  to 
avoid  the  connotation  that  only  non-readers  were  to  be  interviewed,  which 
might  have  distorted  the  results  because  of  respondent  or  interviewer 
reaction  to  this  knowledge.  The  15  students  were  assigned  at  random  to 
the  interview  schedule,  and  the  interviewer  did  not  know  which  were 
readers  and  non- readers. 

c.  Male  and  Female  Written  Survey ' Groups 

The  written  ECHO  survey  was  administered  to  a  class  of  15  male 
students  and  a  class  of  15  female  students.  As  is  shown  in  the  third 
section  of  Table  20,  these  two  incidental  samples  were  similar  in  age, 
years  of  residency  in  the  United  States,  and  education,  but  not  in  marital 
status.  The  purpose  of  these  two  samples  was  to  provide  a  frame  of 
reference  in  which  the  results  from  the  interviewed  group  could  be 
evaluated. 

d.  Limitations  of  Intergroup  Comparability 

The  comparisons  of  written  and  interview  data  are  necessarily  gross, 
and  any  apparent  differences  or  similarities  are  useful  only  to  suggest 
hypotheses  for  further  study,  since  extraneous  differences  between  the 
groups  were  not  controlled  in  sampling.  The  second  section  of  Table  20 
shows  a  marked  difference  in  educational  background  between  the  inter¬ 
viewed  group,  none  of  whom  had  had  education  beyond  the  primary  grades, 
and  the  male  written-survey  group,  67  percent  of  whom  had  gone  beyond 
the  primary  grades;  intergroup  socioeconomic  differences  may  be  inferred 
from  the  educational  difference.  The  most  that  can  be  said  is  that  all 
samples  contained  Spanish-speaking  immigrants,  of  similar  age  and 
residency,  who  were  in  training  at  the  Center. 

e.  Indigenous  Classifier  Teams 

To  perform  classifications,  nine  male  and  three  female  students 
were  randomly  selected  from  a  group  that  had  scored  perfectly  on  the 
literacy  test  but  had  not  participated  in  ECHO  sessions.  These  students 
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served  in  four  three -member  teams  that  respectively  classified  the  data 
from  the  interviewed  non-reader  subgroup,  the  interviewed  reader  sub¬ 
group,  the  male  written  survey  group,  and  the  female  written  survey 
group.  The  three  female  students  classified  the  female  data.  The  three 
students  who  classified  the  interviewed  non-reader  data  had  respectively 
had  six,  six,  and  eight  years  of  education. 


4 .  Spanish-Language  Instructions 

The  E  prepared  simple  English  instructions  for  interviews  and 
group  question  sessions  and  for  classification  sessions.  Two  counselors 
at  the  Skill  Center  translated  these  instructions  into  Spanish.  A  third 
person  then  translated  them  back  into  English,  to  check  accuracy  of 
translation.  The  E  found  some  discrepancies,  and  the  Spanish  versions 
were  corrected.  The  corrected  versions  were  then  translated  into  English 
by  two  new  people  and  found  satisfactory. 


Three  possible  sets  of  ECHO  questions  were  discussed  with  an  inter¬ 
viewer  and  some  respondents  at  another  skill  center  before  the  following 
questions  were  chosen: 


iQue  es  algo  bueno  para  hacer? 
6Quien  lo  aprobaria? 

6Que  es  algo  malo  para  hacer? 
i  Quien  lo  desaprobaria? 


(What  is  a  good  thing  to  do?) 
(Who  would  approve  of  it?) 
(What  is  a  bad  thing  to  do?) 
(Who  would  disapprove  of  it?) 


5 .  Interviews 

All  interviews  were  conducted  individually  by  one  counselor,  who 
came  from  a  different  skill  center  and  consequently  did  not  know  any  of 
the  respondents  personally.  The  same  instructions  were  read  verbatim  to 
each  respondent,  and  the  entire  procedure  was  in  Spanish.  One  interview 

’k 

The  anthropological  technique  of  "back  translation"  has  been  discussed 
by  Phillips  (1959)  . 
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was  observed  by  but  all  others  were  conducted  in  private.  The 
respondents  were  assured  that  their  answers  would  be  treated  anonymously 
and  that  individual  evaluation  was  not  a  purpose  of  the  interview.  No 
attempt  was  made  to  discourage  the  respondents  from  discussing  their 
interview  experience  with  other  students. 

Repetitions  of  the  ECHO  questions  by  the  interviewer  took  the 

form: 

iQue  es  algo  bueno  para  hacer?  (What  is  a  good  thing  to  do?) 

i  Que  es  otra  buena  cosa  para  hacer?  (What  is  another  good  thing 

to  do?) 


Each  interview  required  an  hour  or  more. 


6 .  Indigenous  Classification  Sessions 

The  interviewer  administered  the  indigenous  classification  sessions, 
the  first  two  supervised  by  The  procedure  followed  the  normal  course. 
Though  E  understood  no  Spanish  and  had  to  work  exclusively  through  the 
bilingual  interviewer,  no  difficulty  was  encountered  in  directing  the 
procedure  or  answering  questions  from  the  classifiers. 


The  indigenous  classifiers  worked  slowly  but  maintained  their 
interest  throughout  each  session.  Though  each  classifier  had  about  one- 
third  as  many  cards  to  classify  as  in  other  studies,  the  sessions  lasted 
as  long  as,  or  longer  than,  the  usual  classification  sessions. 


7.  Translation  of  Data 

All  responses  and  indigenous  category  titles  were  translated  into 
English  and  the  translations  reviewed  by  A  display  deck  was  key¬ 
punched  with  English  responses  and  submitted  to  program  PRINDEK,  which 


Responses  were  translated  by  A.  Ordaz;  titles  by  J.J.  Sanchez;  transla¬ 
tions  were  reviewed  by  R.  de  Mille.  Sources  were  classified  by 
R.  de  Mille.  The  professional  classification  team  included  R.  de  Mille 
and  R.P.  Barthol,  assisted  by  M.  Gilson. 


117 


produced  a  reclassification  deck  showing  all  responses  printed  in 
English. 


■k 

8.  Professional  Classification 


a.  Sources 

The  E  classified  all  source  responses  and  assigned  source  code 
numbers . 


b.  Values 

The  E  reviewed  all  of  the  indigenous  category  titles  and  made  up 
a  list  that  included  every  idea  in  the  indigenous  lists.  A  staff 
classifier  sorted  all  responses  into  the  new  categories,  so  far  as  they 
would  fit  easily.  A  second  classifier  reviewed  the  sorting,  added 
cards  to  the  categories,  refined  the  system  of  categories  in  consultation 
with  a  third  classifier,  and  classified  all  remaining  responses. 

9 .  Data  Processing  and  Analysis 

Classification  codes  were  punched  into  the  reclassification  cards 
and  then  combined  with  the  display  deck  information  by  program  NUDEK. 

The  data  were  then  analyzed  by  programs  UNIKOUNT,  PERZPROB ,  and  ROCKEM 
(see  Appendix  III) . 

10.  Results 


a.  Group  and  Subgroup  Differences 

When  groups  and  subgroups  were  compared  on  good  things  to  do,  the 
all-male  interviewed  group  differed  significantly  from  the  male  written- 
survey  group  on  eight  of  the  31  categories.  By  contrast,  the  male  and 


Responses  were  translated  by  A.  Ordaz ;  titles  by  J.J.  Sanchez;  transla¬ 
tions  were  reviewed  by  R.  de  Mille.  Sources  were  classified  by 
R.  de  Mille.  The  professional  classification  team  included  R.  de  Mille 
and  R.P,  Barthol,  assisted  by  M.  Gilson. 
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female  written-survey  groups  showed  no  significant  differences.  The 
interviewed  subgroups,  non-readers  and  readers,  differed  significantly 
on  only  two  categories,  and  so  could  be  treated  together. 

The  comparisons  on  bad  things  to  do  revealed  that  the  interviewed 
group  differed  significantly  from  the  male  written-survey  group  on  six 
of  the  36  categories.  Again  there  were  no  significant  differences 
between  the  male  and  female  written  survey  groups.  The  interviewed  sub¬ 
groups  did  not  differ  significantly. 

When  sources  of  approval  and  disapproval  were  analyzed,  the  inter¬ 
viewed  group  differed  significantly  from  the  male  written-survey  group 
on  six  of  71  source  categories,  while  the  male  and  female  written-survey 
groups  differed  significantly  on  only  one  category.  The  interviewed 
subgroups  did  not  differ  significantly. 

b.  Intergroup  Correlations 

The  Pearson  correlation  between  the  interviewed  group  and  the  male 
written-survey  group  (computed  across  67  good  and  bad  categories)  is 
+0.37  (p  <  0.01);  the  correlation  between  the  male  and  female  written- 
survey  groups  is  +0.58  (p  <  0.01).  The  difference  between  the  two 
correlations  is  significant  at  the  0.05  level,  indicating  a  somewhat 
greater  distance  between  the  interviewed  and  male  written-survey  group 
than  between  the  male  and  female  written-survey  groups. 

c.  Indigenous  and  Professional  Classifications 

The  indigenous  and  professional  classifications  of  100  good  and 
100  bad  responses  from  the  10  interviewed  non- readers  were  compared, 
and  it  was  found  that  74  good  and  62  bad  responses  were  similarly 
classified  by  the  two  teams. 
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d.  Productiveness  of  Interviewed  Respondents 

No  interviewed  respondent  failed  to  answer  any  question.  Pro¬ 
ductiveness  was  similarly  high  in  the  written-survey  groups,  though  a 
few  failures  to  respond  did  occur. 

11.  Discussion 

The  differences  found  between  the  interviewed  and  written-survey 
males  may  have  arisen  from  sample  differences  or  from  treatment  differ¬ 
ences  (that  is,  interview  vs  written  survey)  or  from  both.  Evidence 
has  been  presented  in  Table  20  that  the  interviewed  and  written-survey 
males  were  different  in  educational  and  presumably  social  background, 
whereas  the  two  written-survey  groups  were  similar  in  educational  back¬ 
ground  and  presumably  in  social  background.  In  view  of  this  sampling 
difference,  no  conclusions  can  be  reached  about  the  differential  effects 
of  the  two  treatments,  but  some  hypotheses  to  be  tested  in  future  studies 
are  suggested  below. 

a.  Premise  1 

The  written  survey  is  impersonal.  The  interview  involves  inter¬ 
personal  relations . 

Hypotheses : 

la.  The  interviewer  * s  behavior  or  other  characteristics  may  elicit  a 
consistent  reaction  from  different  respondents,  causing  a  sys¬ 
tematic  bias  in  the  data. 

lb.  Respondents  may  react  more  variably  to  interviewers  than  to  the 
written  task,  causing  the  interview  data  to  be  less  reliable. 

lc.  The  fact  that  the  interviewer  is  waiting  for  an  answer  may  cause 
the  interviewed  respondent  to  answer  after  less  reflection  than 
he  would  in  a  written  survey,  thereby  restricting  the  variety  of 
his  responses. 
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ld.  The  interviewer  will  reinforce  almost  any  response  that  is  given. 
Responses  reinforced  early  in  the  interview  may  tend  to  be 
repeated,  thereby  restricting  the  variety  of  the  responses. 

le.  The  respondent’s  elaboration  of  a  topic  and  his  involvement  in 
communicating  that  topic  to  the  interviewer  may  compete  with  his 
subsequent  engagement  in  a  new  topic,  thereby  restricting  the 
variety  of  the  responses. 

b.  Premise  2 

The  oral  respondent  may  talk  at  length.  The  interviewer,  having 
limited  space,  must  then  condense  the  response,  often  interpreting 
it  or  selecting  a  portion  of  it. 

Hypotheses : 

2a.  Interpretation  or  selection  may  bias  the  data. 

2b.  Selection  or  condensation  may  restrict  the  variety  of  responses. 

2c.  The  interviewer  may  interpret  and  record  as  a  second  response  a 
statement  that  the  respondent  intends  only  as  an  extension  of 
previous  remarks.  The  result  would  be  to  restrict  the  variety  of 
the  responses. 

c.  Premise  3 

The  early  part  of  an  interview  is  spent  in  establishing  rapport 
and  making  the  respondent  comfortable. 

Hypothesis : 

3.  Reactions  to  the  survey  itself  may  be  expressed  during  this  period 
and  therefore  may  not  appear  in  the  data. 

d.  Premise  4 

The  respondent’s  feeling  of  anonymity  is  markedly  greater  in  the 
written  survey. 
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Hypothesis ; 


4.  Hostile  or  antisocial  responses  are  less  likely  to  appear  in  the 
interview  data.  Socially  acceptable  or  conformist  responses  are 
more  likely  to  appear. 

Although  the  present  study  was  not  intended  to  test  these  hypotheses, 
some  of  the  data  are  relevant.  Five  of  the  hypotheses  mention  restriction 
of  the  variety  of  responses.  If  such  restriction  occurred,  it  should 
appear  in  the  data  as  greater  redundancy;  respondents  should  give  more 
repetitive  or  closely  related  responses,  which  should  be  classified  in 
the  same  categories.  Inspection  of  the  data  reveals  no  difference  in 
redundancy  between  the  interviewed  and  written-survey  groups,  and  so 
these  hypotheses  (Ic,  Id,  le,  2b,  and  2c)  receive  no  support. 

Hypothesis  4  predicts  a  greater  tendency  to  conformity  in  the  inter¬ 
view  data.  The  interviewed  group  is  higher  on  bad-thing  categories 
involving  lawbreaking,  taking  drugs,  robbing  and  stealing.  If  this  can 
be  interpreted  as  a  more  conforming  tendency,  some  support  may  be  found 
for  hypothesis  4  in  future  studies. 

12.  Conclusions 


a.  Feasibility  of  ECHO  Interviews 

ECHO  interviews  seem  feasible  as  a  method  of  data  collection. 
Respondents  are  willing  to  answer  questions  and  give  the  required  number 
of  answers.  An  hour  or  more  is  needed  for  each  interview. 

b.  Classification  by  Indigens  with  Limited  Reading  Skill 

The  indigenous  classifiers  in  this  study  were  able  to  carry  out 
the  procedures,  though  they  took  proportionately  two  to  three  times  as 
long  as  other  teams.  Supervision  through  an  intermediary  by  an  ^  who 
does  not  understand  the  language  spoken  by  the  classifiers  is  feasible. 
The  indigenous  classifications  provide  useful  information,  and  agreement 
with  professional  classifications  is  within  expected  limits. 
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c.  Characteristics  of  ECHO  Interview  Data 

Similarity  between  interview  and  written-survey  data  in  this  study 
was  not  sufficient  to  rule  out  bias  due  to  the  interview  technique 
itself. 

F.  STUDENT  COUNCIL  STUDY 

1.  Value  Changes  Resulting  from  Training 

The  original  purpose  of  this  study  was  to  test  whether  ECHO  would 

be  sensitive  to  short-term  changes  in  value  orderings  caused  by  a  brief 
training  program.  The  data  were  collected  and  analyzed,  and  there  were 
some  apparent  changes.  However,  the  intrusion  of  extraneous  activities 
into  the  training  program  rendered  the  outcome  ambiguous  and  uninter¬ 
pretable,  and  so  it  will  not  be  reported. 

Subsequently  the  data  were  analyzed  for  other  purposes,  reported 

below. 


2.  Sample 

Fourteen  male  and  four  female  Student  Council  members  from  the 
University  of  California,  Santa  Barbara  (UCSB),  gave  ECHO  responses  on 
the  first  and  second  day  of  a  two-day  training  program  conducted  at  a 
mountain  lodge.  (These  data  have  not  been  discussed  in  relation  to  the 
other  college  samples,  because  the  orientation  of  the  training  program 
was  markedly  different  from  the  usual  college  background,  and  the  ^s 
were  a  specially  selected  group.) 

3.  Reliability  Analysis 

The  repetition  of  the  question  session  provided  an  opportunity  for 
an  estimate  of  reliability.  After  the  data  from  both  question  sessions 
had  been  combined  and  classified  by  a  team  of  UCSB  students  who  had  not 
been  respondents  in  the  question  sessions,  the  Session-1  responses  were 
separated  from  the  Session-2  responses,  and  response  frequencies  were 
tabulated  for  each  value  category. 
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This  procedure  resulted  in  a  Session-1  distribution  and  a 
Session-2  distribution  across  the  same  30  categories  of  good  things  to 
do,  and  corresponding  distributions  across  19  categories  of  bad  things 
to  do.  Pearson  correlations  were  computed  between  Session  1  and 

Session  2.  These  correlations  appear  in  Table  21  under  the  heading, 
First-Classification  Estimates. 

The  data  were  reclassified  by  another  team  of  non-respondent  UCSB 
students,  and  the  same  steps  were  repeated,  resulting  in  the  Second- 
Classification  Estimates  of  reliability,  shown  in  the  second  part  of 
Table  21.  The  first-  and  second-classification  estimates  are  different 
because  the  two  classification  teams  grouped  the  cards  differently;  they 
are,  however,  based  on  the  same  responses  and  are  not  independent 
estimates . 


TABLE  21 

ECHO  GROUP  RESPONSE  RELIABILITY 


First-Classification  Estimates; 


Goods 

(30  categories) 

.71 

Bads 

(19  categories) 

.72 

Second-Classification  Estimates: 

Goods 

(20  categories) 

.73 

Bads 

(20  categories) 

.84 
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These  estimates,  ranging  from  0.71  to  0.84  (p  <  0.01)  indicate  a 
satisfactory  level  of  reliability.  The  estimates  are  probably  biased 
downward,  since  one  purpose  of  the  training  program  was  to  change  the 
Council  members’  value  priorities. 

The  _E’s  interpretation  of  these  results  is  that  ECHO  ^s  can  be 
expected  to  respond  on  different  occasions  (not  widely  separated)  with  a 
reliability  that  is  probably  underestimated  at  +0.75. 

4.  ECHO  Categories  as  Group  Phenomena 

The  foregoing  reliability  analysis  raises  a  question  about  the 
source  of  the  consistency  in  ECHO  responses  from  one  question  session  to 
another.  Are  the  responses  consistent  because  individual  Ss  give  the 
same  responses  on  two  occasions,  or  because  the  group  gives  the  same 
responses  without  any  necessary  repetition  of  responses  by  individuals? 

This  question  is  important  in  the  understanding  of  ECHO,  because  it 
is  assumed  that  ECHO  measures  values  of  groups.  The  issue  may  be  clarified 
if  we  imagine  two  extreme  cases,  which  would  never  be  observed  in  actual 
research. 

In  each  extreme  case,  we  collect  ECHO  data  from  100  S^s  on  two 
occasions.  Session  1  and  Session  2.  We  classify  Session-1  data  and 
Session-2  data  together,  and  we  inspect  the  resulting  10  value  categories 
to  see  which  individual  ^s  are  represented  in  which  categories  at 
Session  1,  and  which  at  Session  2. 

In  one  extreme  case,  we  find  that  10  different  subsets  of  ^s  are 
represented  in  the  10  categories,  no  ^  being  represented  in  more  than 
one  category.  The  results  for  Session  1  and  Session  2  are  identical 
(perfect  reliability) .  Our  interpretation  is  that  we  have  inadvertently 
sampled  10  different  populations  at  the  same  time. 
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In  the  other  extreme  case,  we  find  that  1  to  50  are  represented 
exclusively  in  the  even-numbered  categories,  and  ^s  51  to  100  are  rep¬ 
resented  in  the  odd-numbered  categories  for  Session  1,  while  the  exact 
opposite  is  true  for  Session  2.  Reliability  here  is  negatively  perfect 
for  individuals  but  positively  perfect  for  the  group — in  other  words, 
each  individual  performs  inversely  in  the  two  sessions,  but  the  group 
performs  identically.  Our  interpretation  is  that  we  have  sampled  one 
homogeneous  population,  whose  stable  group  values  are  expressed  at 
different  times  by  different  members  of  the  group,  but  every  time  by  the 
group  as  a  whole.  A  more  general  interpretation  is  that  ECHO  does 
measure  group  values  and  not  merely  aggregates  of  values  uniquely  held 
by  individuals. 

5.  Respondent  Inconsistency  Analysis 

The  Student  Council  data  were  further  analyzed  to  see  whether  the 
substantial  reliability  of  ECHO  was  a  function  of  group  consistency  or 
of  individual  consistency. 

The  categories  of  good  things  were  put  in  rank  order  by  the  number 
of  ^s  represented  in  them  (a  person  who  contributed  to  the  same  category 
in  both  sessions  was  counted  twice  in  that  category) ;  then  the  category 
ranks  were  used  as  scale  points,  on  a  figure  like  Figure  5,  and  plotted 
against  the  percentage  of  contributors  to  each  category  who  had  con¬ 
tributed  in  one  session  only.  The  categories  of  bad  things  were  treated 
the  same  way. 

The  two  curves,  one  for  goods  and  one  for  bads,  were  very  similar. 
The  good  and  bad  data  were  combined  and  plotted  again,  using  intervals  of 
two  category  ranks  to  smooth  the  curve.  The  result,  in  Figure  5,  shows 
that  in  categories  to  which  less  than  half  the  group  contribute  at  each 
session  (category  size  shown  by  the  dotted  line) ,  only  about  half  of  the 
contributors  are  consistent  from  one  session  to  the  next.  (In  the  more 
important  categories,  to  which  more  than  half  the  group  contribute  at 
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each  session,  individual  inconsistency  is  less  and  less  possible  as  the 
category  size  increases.) 

If  the  reliability  of  ECHO  results  depended  chiefly  on  consistent 
performance  by  individual  respondents,  then  the  inconsistency  curve 
would  not  rise  as  it  does  in  Figure  5.  The  progressively  smaller  cate¬ 
gories,  which,  in  any  one  session,  are  generated  by  progressively  smaller 
subsets  of  ^s,  may  be  legitimately  interpreted  as  pervasive  group  values, 
because  when  the  session  is  repeated,  about  half  of  the  composition  of 
each  small  subset  changes. 

A  hypothesis  that  could  be  tested  is  that  if  several  equivalent 
question  sessions  were  conducted  with  the  same  homogeneous  group,  every 
^  would  in  one  session  or  another  be  represented  in  every  category. 
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6 .  Analysis  of  Importance  Rankings 

Relationships  between  ECHO  category  frequency  ranks  (ranks  based 
on  the  number  of  represented  in  the  category  or  the  number  of  responses 
composing  the  category)  and  importance  rankings  by  classifiers  have  been 
reported  earlier  (Barthol  and  Bridge,  1967).  In  the  Student  Council  Study, 
the  four  teams  of  classifiers  were  asked  to  rank  the  categories  they  had 
produced  according  to  their  feelings  about  how  "important"  each  category 
was.  Spearman  rank  correlations,  r^ ,  (Guilford,  1956)  were  computed 
between  the  frequency  ranks  (based  on  the  number  of  responses  in  each 
category)  and  classifier  rankings.  The  results  are  shown  in  Table  22. 

The  relationships  are  somewhat  inconsistent  but  suggest  that  the  two 
measures  of  importance  can  be  expected  to  be  correlated  in  the  usual 
ECHO  sample. 

The  ]E  concludes  that  the  size  of  an  ECHO  category  does  reflect  a 
variable  of  felt  importance  in  the  respondent  group. 


TABLE  22 

CORRELATION  OF  CATEGORY  FREQUENCY  RANKS  AND 
"IMPORTANCE"  RANKING  BY  CLASSIFIERS 


Firs t“Classif ication  Team: 


Goods 

(30  categories) 

.55 

<0.01 

Bads 

(19  categories) 

.39 

<0.05 

Second-Classification  Team: 

Goods  (20  categories) 

.68 

<0.01 

Bads 

(20  categories) 

.70 

<0.01 
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MANILA  STUDY 


G  • 


1.  Samples 

The  male  were  students  at  the  Philippine  College  of  Arts  and 
Trades,  ages  17  to  21,  almost  all  from  provincial  towns,  mainly  Tagalog 
with  a  few  Ilocano,  Visayan,  Pangasinan,  and  Bicol,  None  were  Chinese. 

The  female  ^s  were  elementary  education  students  at  Philippine  Normal 
College,  ages  17  to  20,  and  like  the  male  ^s  in  origin  and  language. 

After  exclusion  of  10  males  and  10  females  who  had  not  followed  the 
directions,  the  samples  included  27  males  and  39  females. 

The  male  indigenous  classifiers  were  six  Tagalog-speaking,  middle- 
class,  urban  college  students,  ages  18  to  23;  the  females  were  six 
Tagalog-speaking,  middle-class,  urban  research  assistants,  ages  20  to  25. 

2-  Data  Collection  and  Classification 

The  ^s  were  permitted  to  answer  in  English  or  Tagalog;  most  answered 
in  English.  The  classification  of  male  data  was  done  by  males,  female 
data  by  females.  Two  value  classifications  of  each  sample  were  made. 

After  the  sources  were  classified  and  all  Tagalog  responses  were  trans¬ 
lated,  the  data  cards  were  sent  to  the  United  States. 

3.  Data  Processing 

The  indigenous  classification  codes  were  punched,  and  the  data  were 
analyzed  on  program  UNIKOUNT.  A  display  deck  was  prepared. 

4.  Intercultural  College  Study 

The  Manila  samples  were  included  in  the  College  Study,  where  various 
methodological  and  cultural  findings  are  reported. 

* 

The  question  and  classification  sessions  were  supervised  by  G.M.  Guthrie, 
of  Pennsylvania  State  University,  who  selected  the  ^s. 
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TABLE  23 


INFORMATION  LOST  WHEN  10-CARD-PER-^  SAMPLES  ARE  REDUCED  TO  7  CARDS  PER  S 

■k 

Data  Sample 


Number  of  Ss 

F  + 

39 

M  + 

27 

F  - 

39 

M  - 

27 

Sum 

132 

Mean 

33.0 

Categories 

in  10-card  samples 

18 

15 

29 

14 

76 

19.0 

in  7-card  samples 

17 

15 

28 

13 

73 

18.2 

Loss 

1 

0 

1 

1 

3 

0.75 

Percent  Loss 

5.6 

0 

3.4 

7.1 

3.9 

Redundant  Responses 

in  10-card  samples 

95 

108 

86 

90 

379 

in  7-card  samples 

46 

59 

45 

47 

197 

Reduction 

49 

49 

41 

43 

182 

Percent  Reduction 

51.6 

45.4 

47.7 

47.8 

48.0 

Mean  Ss  per  Category 

in  10-card  samples 

15.4 

10.3 

9.9 

10.6 

11.4 

in  7-card  samples 

12.4 

8.6 

7.7 

8.9 

9.2 

Reduction 

3.0 

1.7 

2.2 

1.7 

2.2 

Percent  Reduction 

19.5 

16.5 

22.2 

16.0 

19.3 

Intersample  Correlation 

.97 

.99 

.98 

.97 

** 

.98 

Intersample 

Determination 

.94 

.98 

.96 

.93 

k-k* 

.95 

k 

Female  and  male  groups 

answering 

good  (+) 

and  bad 

(-)  ECHO 

questions . 

Ten-card  sample  correlated  with  7-card  sample  over  frequencies  of  ^s  repre¬ 
sented  in  each  category  (Pearson  r) . 

•kkk  2 

Coefficient  of  determination  (r  )  indicates  the  proportion  of  the  variance 
in  the  10-card  sample  that  is  determined  by  variance  in  the  7-card  sample. 
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5. 


Reduction  of  IQ-Card  Sample  to  7-Card  Sample 

To  test  the  hypothesis  that  the  number  of  ECHO  question  cards  pre¬ 
sented  to  an  ^  could  be  reduced  without  serious  loss  of  information,  the 
Manila  samples  were  reduced  to  seven  cards  per  ^  and  the  information 
loss  evaluated. 

The  Manila  samples  were  chosen  for  this  test  for  the  following 
reasons : 

1.  The  data  had  been  carefully  collected,  from  cooperative  ^s, 
under  good  conditions,  and  ^s  who  had  not  followed  instruc¬ 
tions  had  been  eliminated. 

2.  The  indigenous  classifications  had  produced  well-constructed 
and  sufficiently  numerous  categories. 

3.  The  incidence  of  nonresponse  was  low  (0.55  blank  cards  per  S_ 
for  each  question). 

a.  Procedure 

The  last  three  of  the  10  cards  presented  to  each  ^  were  removed 
from  the  data  decks,  and  the  reduced  7-card  decks  were  processed  on 
program  UNIKOUNT. 

b.  Results 

Table  23  shows  the  results  of  reducing  the  data  by  three  cards  per 
Four  separate  samples  are  analyzed:  female  and  male  good  things, 
and  female  and  male  bad  things.  The  female  and  male  data  are  completely 
independent  because  they  involve  different  ^s;  the  good  and  bad  data  for 
either  males  or  females  are  largely  independent  because  the  ^s  were 
answering  different  questions. 

The  first  line  in  Table  23  indicates  that  no  ^s  were  lost  when  the 
data  were  reduced  (the  numbers  were  the  same  before  and  after  reduction 
and  are  therefore  shown  only  once) . 
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The  second  section  of  Table  23  shows  that  the  reduction  resulted  in 
an  average  loss  of  0.75  categories  per  sample,  or  3.9  percent  of  the 
categories . 

The  third  section  of  the  table  shows  that  redundant  responses  were 
reduced  an  average  of  48  percent.  Redundant  responses  are  responses 
beyond  the  first  response  for  one  S_  in  one  category;  they  are  redundant 
because  once  the  ^  has  been  counted  as  represented  in  that  category  (by 
his  first  response),  he  is  not  counted  again,  no  matter  how  many 
additional  responses  he  may  have  in  the  category. 

The  fourth  section  shows  that  the  mean  number  of  ^s  represented  in 
a  category  was  reduced  by  2.2  ^s,  or  19.3  percent  of  the  mean  number  of 
^s  per  category  in  the  10-card  samples.  In  a  table  showing  percentages 
of  each  group  represented  in  each  category  (the  PERZPROB  table,  for 
example),  the  average  reduction  would  be  6.7  percent  of  the  group. 


The  last  section  of  Table  23  shows  coefficients  of  intersample 
correlation  and  determination.  In  each  column,  the  10-card  sample  and 
the  7-card  sample  were  correlated  across  numbers  of  ^s  represented  in 
each  category.  The  average  intercorrelation  is  0.98.  To  estimate  the 
proportion  of  variance  in  the  10-card  sample  that  is  determined  by  (or 
that  can  be  predicted  from)  variance  in  the  reduced  7-card  sample, 
coefficients  of  determination  were  computed  by  squaring  the  coefficients 
of  correlation  (Guilford,  1956).  The  average  proportion  of  variance  in 
the  10-card  sample  that  is  completely  predictable  from  the  variance  in 
the  7-card  sample  is  0.95. 

c.  Discussion 

The  fact  that  no  ^s  were  lost  is  not  surprising,  since  only  three 
cards  per  ^  were  removed  and  each  ^  had  10  cards  at  his  disposal. 


132 


The  loss  of  less  than  one  category  per  sample  (about  4  percent  of 
the  information),  or  no  more  than  one  category  in  any  of  the  four  samples, 
is  a  small  price  to  pay  for  a  28~percent  reduction  in  the  number  of 
responses  to  be  classified  or  a  reduction  of  30  percent  of  the  number  of 
question  cards  to  be  prepared,  administered,  and  processed.  (The 
difference  here  between  28  percent  and  30  percent  results  from  the  fact 
that  a  few  cards  were  returned  without  responses  written  on  them.)  The 
categories  lost  contain  no  more  than  two  ^s  and  are  at,  or  next  to,  the 
bottom  of  the  hierarchy. 

The  reduction  of  redundant  information  by  48  percent  is  an  improve¬ 
ment  in  the  sample,  particularly  when  requiring  only  a  28-percent  reduc¬ 
tion  of  responses. 

The  6.7-percent  reduction  in  group  percentages  represented  in 
categories  implies  that  percentages  from  7-card  samples  should  not  be 
directly  compared  with  percentages  from  10-card  samples;  the  systematic 
difference  would  spuriously  increase  the  number  of  intergroup  differences 
in  particular  categories.  Another  way  of  looking  at  this  reduction  is 
that  it  constitutes  a  19.3-percent  loss  of  information  about  the  numbers 
of  ^s  represented  in  particular  categories  (less,  it  may  be  noted,  than 
the  28-percent  general  reduction) .  A  third  point  is  that  many  of  the 
categories  in  the  10-card  samples  contain  less  than  6,7  percent  of  the 
respective  groups,  and  yet  only  one  category  tends  to  be  lost  in  each 
hierarchy;  apparently  the  reduction  of  group  percentages  affects  the 
larger  categories  more  than  the  smaller  categories;  this  suggests  that 
^s  tend  to  run  through  their  repertory  of  ideas  in  the  first  seven 
responses . 

The  fact  that  95  percent  of  the  variance  in  the  larger  samples  can 
be  predicted  from  the  variance  in  the  smaller  samples  means  that  we  can 
expect  very  little  difference  in  the  rank  order  of  categories  (computed 
from  numbers  of  ^s  represented  in  categories)  obtained  from  10-card  and 
7-card  samples  (from  the  same  ^s) . 
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d.  Number  of  ECHO  Question  Cards:  Conclusions 

From  the  foregoing  findings,  ^  concludes  as  follows: 

!•  ECHO  samples  obtained  from  7--card  question  sets  are  fully 
satisfactory  when  the  number  of  ^s  is  approximately  30  or 
more. 

2.  Group  percentages  from  7-card  sets  should  not  be  directly 
compared  with  group  percentages  from  10-card  sets,  but  group 
intercorrelations  across  numbers  of  ^s  represented  in  cate¬ 
gories  may  be  interpreted  with  some  confidence  since  95  percent 
of  the  variance  in  a  10-card  sample  is  determined  by  the  first 
seven  cards. 

3.  It  is  better  to  collect  301  responses  from  43  ^s  (seven 
responses  per  than  300  responses  from  30  ^s  (10  responses 
per  . 

4.  In  general,  7-card  question  sets  are  to  be  preferred  over  10- 
card  question  sets. 

H.  OAKLAND  HIGH  SCHOOL  STUDY 

The  Oakland  High  School  Study  was  begun  in  October  1967  with  the 
following  methodological  objectives; 

1.  To  improve  data  collection  procedures  for  larger  samples 
containing  various  subgroups. 

2.  To  use  indigenous  classifiers  in  producing  value  hierarchies 
that  would  be  specific  to  certain  subgroups,  and  others  that 
would  include  all  ^s . 

3.  To  test  computer  programs  being  developed  to  process  larger 
amounts  of  data. 

4.  To  explore  the  sensitivity  of  ECHO  by  trying  to  distinguish 
among  certain  subgroups  through  validity  checks  or  message 
sessions • 
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1.  Sample 

The  S_s  were  180  students  at  a  high  school  in  Oakland,  California. 
Subgroupings  used  during  the  data  analysis  were  by: 


1. 

Ability 

Gifted  and  disadvantaged 

2. 

Grade 

10th,  11th,  and  12th  grades 

3. 

Sex 

Male  and  female 

4. 

Origin 

White,  black,  Mexican,  oriental 

All  data  were  collected  in  classrooms. 

2.  Classification 

Twelve  teams  of  students  classified  the  data  from  their  own  groups 
(two  lOth'-grade,  two  llth-grade,  and  two  12th-grade  classes)  ;  each  team 
classified  both  goods  and  bads  (separately) ,  so  that  all  data  were 
classified  twice.  Later,  two  teams  classified  all  of  the  data  into  one 
inclusive  system  of  good  things  to  do  and  another  of  bad  things  to  do. 

3.  Data  Analysis 

Value  and  source  hierarchies  were  derived  for  each  class  group  and 
for  the  total  sample.  The  hierarchies  were  based  on  response  counts 
rather  than  ^  counts.  All  results  were  tabulated  and  systematically 
inspected. 

The  source  data  were  used  to  test  two  hypotheses: 

1.  Negro  ^s  will  mention  Father  as  a  source  less  often  than 
white  ^s . 

2.  Disadvantaged  ^s  will  mention  Police  more  often  than  gifted 
S_s. 

Neither  of  these  hypotheses  received  any  support. 

R.  Hudson,  of  the  school  staff,  helped  to  organize  and  conduct  the 
question  and  message  sessions. 
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Extensive  analysis  was  carried  out  by  means  of  program  ROCKEM, 
comparing  different  classifications  of  the  same  responses  to  estimate  the 
degree  of  congruity  (or  reliability)  of  indigenous  classifications.  In 
general,  the  results  showed  that  the  few  larger  categories  corresponded 
across  classifications  at  a  nonchance  level,  but  the  smaller  categories 
did  not. 

4.  Message  Sessions 

The  value  statements  generated  by  the  indigenous  classifiers  were 
used  to  construct  messages  (lists),  intended  to  appeal  specifically  to 
various  subgroups.  Each  class  was  given  several  paired  lists,  with 
instructions  to  select  the  list  that  most  nearly  described  the 

The  results  were  inconclusive.  While  many  ^s  selected  the  messages 
generated  from  their  class  data,  others  selected  messages  from  a  different 
group.  Some  results  were  easily  explained  (for  example,  lOth-grade  gifted 
^s  selected  the  messages  designed  for  12th-grade  gifted) ;  other  results 
were  not  explainable  (such  as  llth-grade  disadvantaged  ^s  selected  10th- 
grade  gifted  "bads,"  which  the  lOth-grade  gifted  ^s  rejected). 

5.  Discussion 

Four  methodological  findings  emerged  from  this  study: 

1.  Indigenous  classifiers  are  very  useful  and  sometimes 
indispensable . 

2.  Anomalies  in  the  data  are  relatively  easy  to  identify. 

3.  Responses  may  reveal  much  about  the  environment  in  which  the 
^s  live. 

4.  ECHO  data  analysis  techniques  were  not  adequate  to  make  good 
use  of  the  data. 
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As  an  example  of  the  need  for  indigenous  classifiers,  inspection  of 
a  category  titled  "Have  sexual  relations"  revealed  the  following  four 
responses,  among  others:  "Go  down  to  7th  Street  when  the  sale  is  on," 

"Pull  train,"  "Keep  the  faith,  baby,"  and  "Be  a  hoe."  The  classifiers 
were  black  female  12th  graders.  The  ^’s  interpretations  of  two  of  the 
responses  were  confirmed  by  the  classifiers:  "Be  a  hoe"  was  the  phonetic 
rendition  of  "Be  a  whore";  "Pull  train"  was  slang  borrowed  from  white 
motorcycle  clubs  signifying  a  low-status  female  member  of  a  group  accepting 
sexual  relations  with  all  of  the  males  in  the  group.  The  other  two 
responses  had  to  be  clarified  by  the  classifiers:  on  Thursday  evenings, 
the  day  before  payday,  the  entrepreneurs  of  the  local  red-light  district, 
7th  Street,  cut  rates  and  the  sale  is  on;  "Keep  the  faith,  baby"  the 
classifiers  declared,  was  local  slang  for  intercourse. 

As  an  example  of  an  anomaly  in  the  data,  inspection  of  a  category 
called  "Play  with  sex,"  which  comprised  mostly  expected  responses  (necking, 
petting,  drive-in  movies,  etc.),  also  included  several  about  rape.  Since 
the  act  of  rape  seemed  strong  for  a  category  about  playing  with  sex,  ^ 
suspected  some  misunderstanding.  A  class  of  18  disadvantaged  12th 
graders  were  asked  by  the  teacher  to  define  "Rape."  Eight  responses 
referred  to  intercourse  by  force  ("My  definition  would  be  a  girl  being 
taken  advantage  of  sexually.  This  happens  often,  but  the  girl  seldom 
yells  rape  until  it’s  over.");  three  were  unclassif iable  ("Like  you  rape 
up  some  leaves  that  fall  from  a  tree");  and  seven  defined  rape  as 
being  denuded  by  force  ("To  take  clothes  off  someone  without  their 
consent") . 

None  of  the  teachers  were  aware  of  this  widespread  nonstandard 
usage;  inquiry  among  professionals  working  with  disadvantaged  groups 
failed  to  reveal  one  who  was  familiar  with  it.  This  is  an  example  of 
echo’s  high  informational  utility  (see  Milburn,  Barthol,  and  de  Mille, 
1968). 
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ECHO  responses  reveal  much  about  the  environment  in  which  the 
live.  Up  to  this  time,  the  samples  had  been  almost  entirely  white  middle 
class;  virtually  no  one  had  said  "A  bad  thing  to  do  is  be  a  prostitute"; 
yet  this  was  a  frequent  response  from  the  black  disadvantaged  students. 

It  was  easy  to  ascertain  that  prostitution  was  common  in  the  neighborhoods 
in  which  these  students  lived;  the  teachers  believed  that  some  of  the 
girls*  female  relatives  were  prostitutes.  Middle-class  girls  usually  do 
not  consider  prostitution  to  be  a  family  problem;  it  is  not  salient  and 
rarely  mentioned.  The  ^  concluded  that,  despite  social  disapproval,  a 
bad  thing  mentioned  by  respondents  did  exist  in  their  culture  sufficiently 
to  cause  them  concern  (though  lack  of  mention  would  not  guarantee  absence 
of  the  bad  thing) .  Thus  UCLA  junior  and  senior  college  students  mentioned 
marijuana;  Northwestern  freshmen  did  not;  marijuana  is  thought  to  be  used 
more  at  UCLA  than  at  Northwestern. 

The  two  main  weaknesses  revealed  in  ECHO  data  analysis  were  the 
rather  poor  quality  of  indigenous  classifications  and  the  insufficient 
power  of  the  computer  programs  to  count  the  most  useful  observations  (that 
is,  ^s  represented  rather  than  responses  included  in  categories)  and 
perform  statistical  tests  of  ECHO  results.  These  findings  led  to  later 
modifications  in  the  relation  of  indigenous  to  staff  classification, 
and  to  the  development  of  the  computer  programs  UNIKOUNT  and  PERZPROB . 

The  failure  of  the  messages  to  discriminate  consistently  among  the 
subgroups  might  have  resulted  from  poor  indigenous  classifications  and 
titling,  from  insufficient  value  differences  between  the  subgroups,  or 
from  other  causes  unknown. 

I.  ARROWHEAD  TRAINING  GROUP  STUDY 

The  Arrowhead  Study,  begun  in  September  1967,  was  designed  to 
identify  changes  in  values  resulting  from  a  training  program.  In  addition, 
a  second  form  of  the  ECHO  question  was  tested:  **What  is  a  good/bad  thing 
to  happen?" 
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1.  Sample 

The  were  90  participants  in  a  sensitivity  training  program, 
randomly  assigned  to  six  training  groups. 

2.  Study  Design  and  Actual  Procedures 

Data  were  to  be  collected  at  the  beginning  and  end  of  the  training 
period,  from  trainees  and  from  a  control  group  not  undergoing  the 
training.  Data  were  collected  from  the  trainees  and  were  classified  in 
group  samples  by  indigens  and  in  a  combined  system  by  ECHO  staff  classi¬ 
fiers.  Violating  the  design,  however,  the  second  data  samples  had  to  be 
collected  by  mail,  and  40  percent  of  the  ^s  failed  to  return  their 
packets  even  after  two  reminders  were  sent  out. 

This  turn  of  events  presented  an  opportunity  for  comparing  values 
in  two  groups  of  known  behavioral  differences:  returners  and  nonreturners 
Mail-surveyors  commonly  believe  that  nonreturners  are  generally  different 
from  returners.  Surprisingly,  the  ECHO  analysis  resulted  in  virtually 
identical  value  hierarchies:  only  one  category  out  of  107  was  signifi¬ 
cantly  different  (p  <  0.05)  in  the  two  groups. 

Some  minor  methodological  differences  between  the  first  data 
collection  in  this  study  and  data  collections  in  previous  studies  should 
be  noted.  Instructions  were  written  instead  of  oral;  packets  were 
individually  handed  to  ^s,  who  were  allowed  to  complete  the  responses  at 
their  convenience;  and  ^  had  relatively  little  control  over  the  ^s. 

Allowed  so  much  freedom,  10  percent  of  the  prospective  ^s  did  not  return 
their  data  packets. 

The  new  form  of  the  ECHO  question,  "What  is  a  good/bad  thing  to 
happen?"  was  found  to  provide  data  that  were  generally  interpretable  and 
not  inferior  to  the  data  provided  by  good/bad- thing- to-do  questions. 

This  finding  led  to  the  use  of  the  happen  questions,  somewhat  modified, 
in  the  Pen tali th  Study. 


139 


3. 


Clinical  Use  of  ECHO  Results 


While  examining  the  indigenous  value  hierarchies  of  the  various 
subgroups,  ^  was  struck  by  the  impression  that,  of  the  three  groups  that 
had  received  the  good/b ad- thing- to-do  questions,  Group  3  was  different 
from  Groups  1  and  2  and,  in  fact,  appeared  to  be  different  from  most 
groups  that  participate  in  this  kind  of  self-development  program. 

Table  24  shows  some  of  the  data  that  caused  ^  to  infer  that  Group  3  was 
joyless,  unable  to  accept  personal  closeness,  and  alienated  from  natural 
bodily  experience. 

The  Inclusive  Title  in  Table  24  is  the  title  for  a  category 
including  responses  from  all  three  groups.  The  same  responses  were 
classified  by  the  indigenous  classifiers  under  category  titles  peculiar 
to  each  group.  The  first  part  of  the  table  shows  the  representation  of 
each  group  in  the  inclusive  category,  ’’Participate  in  Recreational 
Activities.”  Groups  1  and  2  have  responses  that  were  classified  mainly 
in  categories  involving  enjoyment,  but  Group  3  has  relatively  few 
responses,  and  they  were  classified  mainly  under  "exercise.”  The  second 
inclusive  title,  "Enjoy  Sexual  Activities,”  corresponds  to  self-indulgence, 
enjoyment,  love,  and  sex  in  Groups  1  and  2,  but  from  Group  3  there  is 
only  one  response,  which  was  classified  by  the  indigenous  team  under 
"Respond  to  Conditioned  Reflexes.” 

These  intergroup  differences  are  striking,  and  there  were  others. 
After  studying  all  of  the  comparisons,  the  E  hypothesized  that  Group  3 
was  humorless,  emotionally  blocked ,. tense ,  self-serving,  and  inter- 
personally  handicapped.  These  inferences  were  confirmed  by  the  director 
of  the  training  program,  who  reported  that  the  Group  3  trainer  (an 
experienced  professional)  had  complained  that  his  group  were  inhibited, 
dysphoric,  and  unproductive.  The  trainer,  in  turn,  was  gratified  to 
learn  that  a  disinterested  observer  had  evidence  to  support  his  point  of 
view,  which  he  had  feared  might  be  only  a  realization  of  his  failure  to 
lead  the  group  effectively. 
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TABLE  24 


SELECTED  VALUE  DIFFERENCES  IN  THREE  TRAINING 

n  Group  1  Titles  n  Group  2  Titles  n 

Inclusive  Title:  Participate  in  Recreational  Activities 

9  Indulge  self  12  Enjoy  personal  5 

recreational  pleasures 

3  Enjoy  nature  3  Enjoy  beauty  1 

2  Improve  self 

1  Enjoy  friends 

Inclusive  Title:  Enjoy  Sexual  Activities 
5  Indulge  self  3  Have  love  and  sex  1 

1  Enjoy  friends  1  Care  for  family  and 

children 

* 

Number  of  responses. 


GROUPS 


Group  3  Titles 


Engage  in  physical 
exercise 

Respond  to 

conditioned 

reflexes 


Respond  to 

conditioned 

reflexes 
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This  inductive  use  of  ECHO  categories  requires  a  background  of 
clinical  experience  and  is  not  considered  a  primary  use  of  ECHO.  However, 
it  illustrates  very  well  the  high  informational  utility  of  ECHO,  since 
the  finding  was  wholly  unexpected,  was  confirmed,  and  proved  useful. 

J.  HIGH  SCHOOL  TRAINING  GROUP  STUDY 

This  study  was  designed  to  identify  changes  in  values  resulting 
from  a  training  program.  In  addition,  the  _Ss  were  asked  to  respond  not 
as  themselves  but  in  a  special  role  of  "villager."  The  questions  were: 

"You  are  a  villager.  What  is  a  good  [bad]  thing  for  you  or  a  villager 
like  you  to  do?"  The  ^s ,  American  high  school  students,  were  to  be 
tested  before  and  after  training  for  public  health  field  work  in 
Guatemala  and  Nicaragua,  and  again  after  returning  from  the  field.  It 
was  predicted  that  the  values  of  the  S_s  would  progressively  become  more 
like  those  of  ^s  who  had  already  been  through  the  program  and  more  like 
values  of  the  field  cultures  as  specified  by  experts. 

After  the  first  data  collection  and  classification  (before  training), 
permission  to  continue  the  study  was  withdrawn  by  the  administrators  of 
the  training  program.  The  data  were  used  for  other  methodological  pur¬ 
poses,  described  below. 

1.  Sample 

The  ^s  were  50  male  and  female  high  school  students  participating 
in  the  training  program.  The  question  session  was  administered  to  all 
^s  at  the  same  time,  in  the  usual  way. 

2 .  Classifications 

All  responses  were  classified  twice,  in  one  day.  Male  responses 
were  classified  by  two  teams  of  nonrespondent  male  indigens,  teams  MA 
and  MB;  female  responses  by  corresponding  teams,  FA  and  FB.  In  the  first 

« 

Z.  Pazmany  contributed  to  this  research  design  and  helped  with  the 
selection  of  Ss  and  the  data  collection  and  classification. 
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session,  the  A  teams  classified  goods  while  the  B  teams  classified  bads; 
in  the  second  session,  the  A  teams  sorted  the  bads  while  the  B  teams 
sorted  the  goods,  as  described  in  the  next  paragraph. 

3 .  Structured  Sorting 

To  estimate  the  reliability  of  the  classifications,  each  team  was 
asked  to  sort  the  cards  classified  by  the  other  same-sex  team  under  the 
category  titles  that  had  been  assigned  by  the  other  team.  There  were  no 
clues  as  to  how  the  other  team  had  grouped  the  cards;  the  sorters  had 
only  the  category  titles  and  the  responses  to  work  with. 

The  percentages  of  cards  identically  sorted  were:  60,  69,  66,  and 
71  percent  for  male  goods,  male  bads,  female  goods,  and  female  bads, 
respectively.  This  quite  consistent  result  reflects  a  fairly  low  relia¬ 
bility  of  classification,  which  is  to  be  expected  from  untrained 
classifiers.  Such  errors  of  sorting  are  thought  to  arise  chiefly  from 
defects  in  the  original  classification,  where  categories  may  be  poorly 
formed  or  vaguely  titled,  or  where  some  very  similar  titles  may  coexist 
(for  example.  To  be  discourteous.  To  be  disrespectful). 

4.  Classification  of  Rokeach  Values 

Before  the  classification  sessions,  the  ^  inserted  into  each  data 
sample  18  cards  bearing  bogus  responses  that  were  in  fact  paraphrases 
of  18  values  and  18  disvalues  used  by  Rokeach.  The  responses  are 
listed  in  Table  25;  each  response  was  handwritten  on  a  response  card  and 
accompanied  by  a  plausible  source  response  (such  as  myself,  my  parents, 
the  village) . 


This  test  had  been  suggested  by  T.W.  Milburn  and  had  been  used  once  in 
the  Pentalith  Study,  where  results  were  very  similar.  Rokeach's  research 
is  discussed  by  Milburn,  Barthol,  and  de  Mille  (1968,  pp.  11-12). 
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TABLE  25 


PARAPHRASES  OF  ROKEACH  INSTRUMENTAL  VALUES 


Good  Things  To  Do 

1.  Work  hard  and  aspire  ambitiously. 

2.  Be  open  minded  and  tolerant. 

3.  Perform  competently  and  effectively  at  everything  I  do. 

4.  Be  cheerful  and  joyful. 

5.  Be  neat  and  tidy. 

6.  Act  bravely  and  stand  up  for  my  beliefs. 

7.  Forgive  others. 

8.  Be  helpful  and  work  for  the  welfare  of  others. 

9.  Act  sincerely  and  honestly. 

10.  Be  imaginative  and  daring. 

11.  Act  self~reliantly  and  independently. 

12.  Act  intelligently  and  reflectively. 

13.  Act  in  a  consistent,  rational,  logical,  controlled  fashion. 

14.  Be  loving  and  affectionate. 

15.  Act  in  a  dutiful,  respectful,  obedient  manner. 

16.  Act  politely  and  courteously. 

17.  Act  dependably  and  responsibly. 

18.  Act  in  a  restrained,  self-disciplined  or  self-controlled  way. 


Bad  Things  To  Do 

1.  Be  lazy  or  abandon  my  goals. 

2.  Behave  intolerantly  toward  others. 

3.  Act  incompetently  or  ineffectually. 

4.  Behave  in  a  sad  and  dismal  way. 

5.  Be  slovenly. 

6.  Act  cowardly. 

7.  Act  in  an  unforgiving  fashion  toward  others. 

8.  Ignore  others  who  need  help. 

9.  Lie  or  be  insincere. 

10.  Be  unimaginative,  constricted,  or  closed  up. 

11.  Depend  upon  others  instead  of  relying  on  ones  self. 

12.  Act  stupidly. 

13.  Be  illogical  or  inconsistent. 

14.  Act  hatefully  or  harshly  toward  others. 

15.  Act  disobediently  or  disrespectfully  towards  persons  in  authority. 

16.  Act  discourteously  or  illmanneredly . 

17.  Act  irresponsibly  or  be  undependable. 

18.  Act  in  an  unrestrained  and  undisciplined  fashion. 
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The  purpose  of  this  test  was  to  try  the  Rokeach  values  as  markers 
or  reference  points  in  a  value  system  elicited  by  ECHO.  Since  Rokeach 
intended  his  listed  items  to  represent  different  categories  covering  a 
wide  range  of  values,  it  might  turn  out  that  the  Rokeach  cards  would  be 
distributed  widely  through  an  ECHO  value  hierarchy;  on  the  other  hand,  if 
the  Rokeach  values  were  more  specific  and  less  differentiated  than  ECHO 
value  categories,  they  might  coalesce  into  one  or  two  ECHO  categories. 

a.  Results 

Table  26  shows  how  the  Rokeach  cards  were  distributed  in  the  eight 
classifications.  The  first  line  in  the  table  shows  that  in  the  classifi¬ 
cation  of  male  goods  (+)  by  team  A,  the  18  Rokeach  cards  were  put  into 
only  six  of  the  20  categories,  11  cards  being  put  into  one  category 
titled  "Improve  oneself  by  following  a  good  set  of  ideals." 

The  tendency  of  the  Rokeach  cards  to  be  widely  or  narrowly  dis¬ 
tributed  is  rather  inconsistent.  The  widest  distribution  is  in  classi¬ 
fication  MA-,  where  12  categories  had  Rokeach  cards;  the  narrowest  are 
in  classifications  FA+  and  FB+,  where  only  three  categories  had  Rokeach 
cards.  The  mean  tendency,  shown  in  the  last  row  of  the  table,  is  for  the 

•k 

18  Rokeach  cards  to  fall  into  about  seven  categories. 

b.  Conclusion 

The  ^  concludes  that  this  technique  does  not  provide  reliable 
information  on  the  relation  of  Rokeach  values  to  ECHO  value  categories. 


These  results  are  congruent  with  results  in  the  Pentalith  Study  (de  Mille 
and  Barthol,  1969),  where  the  Rokeach  cards  were  inconsistently  dis¬ 
tributed.  The  40  blockmen's  classification  team,  for  example,  put  nearly 
all  of  their  bad  Rokeach  cards  (along  with  34  genuine  responses)  into  a 
category  they  titled  "Odd  Ball."  Other  teams  distributed  the  cards  some¬ 
what  more  widely. 
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TABLE  26 


DISTRIBUTIONS  OF  ROKEACH  CARDS  IN  ECHO  CLASSIFICATIONS 


Categories  Largest 

Number  of  Including  Set  of  Title  Associated 

Classification  Categories  Rokeach  Cards  Rokeach  Cards  With  Largest  Set 


MA  + 

20 

6 

11 

Improve  oneself  by  following 
a  good  set  of  ideals 

MB  + 

20 

8 

6 

Have  a  good  state  of  mind 

FA  + 

17 

3 

16 

Have  good  standards  of 
living  and  morals 

FB  + 

17 

3 

15 

Conduct  yourself  properly 

MA  - 

27 

12 

3 

3 

Be  irresponsible 

Be  hostile  and  intolerable 

MB  - 

27 

8 

4 

Be  discourteous  toward 
others 

FA  - 

18 

7 

6 

Have  bad  attitudes 

FB  - 

18 

7 

7 

Be  irresponsible 

Mean  + 

18.5 

5.0 

12.0 

Mean  - 

22.5 

8.5 

5.0 

Mean  Total 

20.5 

6.75 

8.5 

M  =  Male 

B  =  Team  B 

F  =  Female 

+  =  Good 

A  =  Team  A 

-  =  Bad 
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K.  SORORITY  SOCIALIZATION  STUDY 

Earlier  ECHO  socialization  studies  have  been  reported  by  Barthol 
and  Bridge  (1967) .  The  Sorority  Study  was  conducted  at  the  University 
of  California,  Los  Angeles,  by  Bridge  and  Heller  (ECHO  Project  staff 
members),  who  hypothesized  that  new  sorority  members  (pledges)  when 
exposed  for  a  while  to  the  sorority  subculture,  would  adopt  the  values 
of  the  old  members  (actives) .  (The  ]Es  defined  socialization  without 
reference  to  any  particular  stage  of  development.) 

1.  Samples 

The  ^s  were  10  pledges  and  20  actives  in  a  sorority  that  included 
46  undergraduate  women.  Value  hierarchies  were  obtained  from  the  ^s  on 
two  occasions,  six  months  apart. 

2 .  Treatment  and  Findings 

Spearman  rank  correlations  (Guilford,  1956)  were  computed  (across 
frequencies  of  responses  in  22  categories  of  good  things  to  do)  between 
each  of  six  pairs  of  groups  and  occasions.  Using  the  cross-lagged  panel 
correlation  design  (Rozelle  and  Campbell,  1969),  the  _Es  found  that  the 
values  of  the  actives  remained  stable  while  the  values  of  the  pledges 
became  more  like  those  of  the  actives  during  the  six-month  period. 

L.  INCOMPLETE  STUDIES 

1.  Watts  Study 

The  purpose  of  this  study  was  to  test  the  feasibility  of  collecting 
ECHO  data  from  disadvantaged  blacks  where  ^  had  little  control  over  the 
prospective  ^s.  A  staff  member  of  a  training  center  in  Watts  distributed 
ECHO  packets  to  young  men  who  attended  the  center;  almost  all  accepted 
the  packets,  but  many  returned  them  uncompleted. 

Inspection  of  the  completed  cards  revealed  a  very  low  level  of 
literacy,  and  it  was  concluded  that  many  of  the  young  men  probably  had 
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not  been  able  to  understand  the  instructions.  Discussion  with  the 
training  staff  member  led  to  the  conclusion  that  some  of  the  men 
probably  felt  too  much  resentment  or  hostility  to  cooperate,  although 
they  had  pretended  to  cooperate  by  accepting  the  packets.  The  study  was 
terminated. 

The  problem  of  surveying  a  hostile  population  was  not  further 
addressed  in  Project  ECHO,  but  it  is  a  problem  well  worth  solving,  since 
the  need  to  understand  the  values  of  dissident  elements  in  society  is 
acute.  The  problem  of  surveying  semiliterates  was  addressed  in  the  East 
Los  Angeles  Skill  Center  Study  (see  also.  Bridge  and  Heller,  1968). 

2.  Leeds  Study 

ECHO  data  were  collected  from  participants  in  a  management  training 
program  conducted  under  the  auspices  of  the  University  of  Leeds  (England) . 
The  purpose  of  the  study  was  to  show  effects  of  the  training  program. 

Since  only  13  Ss  completed  all  of  the  materials,  the  data  were  not  con¬ 
sidered  reliable  enough  for  a  study  of  value  change. 

3.  Hoover  High  School  Study 

ECHO  data  were  collected  from  112  students  at  Hoover  High  School, 

Los  Angeles.  Two  purposes  were  to  provide  training  in  data  collection 
for  ECHO  staff  members  and  to  increase  the  ECHO  data  base  early  in  the 
project.  The  data  turned  out  to  be  unreliable  because  of  unplanned 
variations  in  the  instructions  to  the  ^s .  The  study  was  terminated. 

4.  Fraternity  Study 

A  study  similar  to  the  Sorority  Study  was  undertaken  at  a  fraternity 
at  the  University  of  California,  Los  Angeles.  Data  were  collected  from 
35  fraternity  members.  Inspections  of  the  data  revealed  gross  failures 
to  follow  instructions,  which  were  attributed  to  poor  conditions  during 
the  question  session  (such  as  distractions  and  interruptions).  The  study 
was  terminated. 
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APPENDIX  III 


ECHO  COMPUTER  PROGRAMS 


A.  GENERAL 

Five  computer  programs  developed  in  the  ECHO  project  perform  the 
essential  data  processing  indicated  by  in  Figure  6. 

1.  UNIKOUNT 

Program  UNIKOUNT  counts  responses  in  each  ECHO  category  and 
respondents  having  at  least  one  response  in  each  category;  computes  per¬ 
centages  and  ranks;  and  prints  tables  of  rank-ordered  categories  with 
their  titles.  The  typical  use  of  program  UNIKOUNT  is  in  the  step  labelled 
Separation  into  Subgroups. 

2.  PERZPROB 

Program  PERZPROB  computes  the  probability  of  observing  different 
percentages  of  two  groups  of  respondents  represented  (by  their  responses) 
in  a  category;  and  prints  tables  of  percentages  and  probabilities.  The 
typical  uses  of  program  PERZPROB  are  at  the  points  labelled  Statistical 
Tests  and  Data  Display. 


3.  ROCKEM 

Program  ROCKEM  compares  two  ECHO  classifications  of  the  same 
responses;  prints  tables  showing  response  frequencies  and  category  titles 
from  one  classification  that  are  associated  with  response  frequencies 
and  category  titles  from  the  other  classifications;  and  can  also  list  the 


Other  programs,  used  for  correlation  and  factor  analysis  and  not 
developed  in  Project  ECHO,  will  not  be  discussed.  Technical  support  in 
developing  the  ECHO  computer  programs  was  provided  by  J.A.  Boucher, 

J.T.  Bowman,  A.W.  Mclver,  and  D.L.  Rourke. 
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- 

STAFF 
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(TRANSLATION) 

PREPARATION 
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RATIONAL  ANALYSIS, 

A 
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1 
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- 
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a. 
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b. 
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c. 

FACTOR  ANALYSIS 

d. 
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e. 
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SUBJECT  GROUPING  © 

.(C)@l 


a.  VERIFY,  REFUTE.  OR  QUALIFY 
FINDINGS  BY  COMPARISON  WITH 
EXPERT  OPINION,  INDEPENDENT 
OBSERVATIONS,  READING 

b.  TEST  HYPOTHESES  ^ 
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a. 
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b. 

PREDICT 

c. 

TRAIN 

d. 
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e. 
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LEGEND:  ©  KEYPUNCHING  OR  EDP  REQUIRED 
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©  PRESENT  CLERICAL,  FUTURE  COMPUTER  OPERATION 
©  SKILLED  PROFESSIONAL  REQUIRED 


I 


j 


Figure  6.  The  ECHO  Method,  1968 
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AN-11  ^r.7 


data  cards*  Typically,  program  ROCKEM  is  used  after  an  indigenous  and 
a  professional  classification  have  been  made  of  the  same  responses. 

4.  PRINDEK 

Program  PRINDEK  prints  cards  for  classification  or  reclassification 
to  facilitate  classification  of  translated  data  or  unbiased  combination 
of  disparate  data  samples.  The  typical  use  of  program  PRINDEK  is  in  the 
step  labelled  Preparation  of  Reclassification  Deck. 

5.  NUDEK 

Program  NUDEK  merges  reclassification  codes  with  previous  codes  and 
punched  responses.  The  typical  use  of  program  NUDEK  is  after  classifica¬ 
tion  of  cards  printed  by  program  PRINDEK,  to  combine  the  new  and  old 
classification  information  on  the  same  set  of  data  cards. 

B.  PROGRAM  UNIKOUNT 

Program  UNIKOUNT  counts  responses  in  each  category  and  respondents 
having  at  least  one  response  in  each  category. 

UNIKOUNT  aggregates  all  occurrences  of  each  ECHO  category  number 
punched  in  a  specified  field  in  the  data  cards,  to  arrive  at  the 
response  frequency  for  each  category  (one  card  per  response) .  The  pro¬ 
gram  computes  the  percentage  that  each  response  frequency  represents  of 
the  total  number  of  responses  in  the  analysis.  The  categories  are 
ordered  by  card  frequency,  and  rank  numbers  are  computed,  rank  number  1 
(or  lowest  tie  number)  being  assigned  to  the  category  having  the  most 
cards.  A  rank-ordered  table  is  printed,  each  row  showing  a  category 
number,  and  the  card  frequency,  percentage,  rank  number,  and  title  for 
that  category.  Additional  information  associated  with  the  table  includes 
(1)  a  main  title  (for  example,  "Ravipan  Computer  Class") >  (2)  specifica¬ 
tion  of  reference  field,  that  is,  the  number  of  the  data  field  from 
which  the  information  came,  (3)  total  number  of  ECHO  data  cards  input, 

(4)  number  of  error  cards,  if  any;  that  is,  cards  having  no  category 
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TABLE  27 


EXAMPLE  OF  UNIKOUNT  OUTPUT 


RAVIPAN  COMPUTER  CLASS:  GOODS 

REFERENCE  FIELD  -  2 

CATEGORY  RAW  FREQUENCY 

FEMALES  GRP  X 

PERCENT  SUBJECTS 

RANK 

CATEGORY  TITLE 

4 

5 

100.0 

1.0 

TO 

HELP  OTHERS 

1 

4 

80.0 

2.0 

TO 

TREAT  OTHERS  WITH  RESPECT 

3 

3 

60.0 

3.5 

TO 

BE  A  GOOD  RELATIVE 

7 

3 

60.0 

3.5 

TO 

SUPPORT  THE  NATION 

2 

2 

40.0 

7.5 

TO 

FOLLOW  THE  RULES 

5 

2 

40.0 

7.5 

TO 

HAVE  FRIENDS 

6 

2 

40.0 

7.5 

TO 

LEARN 

8 

2 

40.0 

7.5 

TO 

WORK 

11 

2 

40.0 

7.5 

TO 

ENGAGE  IN  ATHLETIC  ACTIVITY 

15 

2 

40.0 

7.5 

TO 

READ 

9 

1 

20.0 

11.5 

TO 

BE  ECONOMICALLY  SECURE 

14 

1 

20.0 

11.5 

TO 

TRAVEL 

NUMBER  OF  SUBJECTS  =  5 
NUMBER  OF  EXCLUDED  CARDS  =  15 
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number  punched  in  the  specified  data  field,  and  (5)  cards  remaining  in 
the  analysis  after  deletion  of  the  error  cards. 

The  second  UNIKOUNT  table  (see  Table  27)  shows,  for  each  category, 
the  number  of  respondents  (subjects)  having  at  least  one  response  in 
that  category.  To  arrive  at  this  information,  UNIKOUNT  excludes  any 
redundant  cards  for  each  respondent  in  the  category;  that  is,  one  card 
is  excluded  every  time  the  unique  identification  number  of  the  respondent 
is  found  to  be  repeated  in  one  category.  The  program  computes  the  per¬ 
centage  that  the  resulting  frequency  represents  of  the  total  number  of 
respondents  found  in  the  sample.  The  other  operations  are  like  the 
operations  for  the  first  table.  Additional  information  includes: 

(1)  main  title,  (2)  reference  field,  (3)  number  of  subjects  (respondents), 
and  (4)  number  of  excluded  cards. 

C.  PROGRAM  PERZPROB 

Program  PERZPROB  computes  the  probability  of  observing  different 
percentages  of  two  groups  of  respondents  represented  in  an  ECHO  category. 

The  program  prints  a  percentage  table  (Table  28)  showing  categories 
as  rows  (with  or  without  titles)  and  groups  of  respondents  as  columns. 

Each  cell  of  the  table  shows  the  percentage  of  one  group  of  respondents 
(subjects)  that  were  represented  in  one  ECHO  category. 

The  program  prints  a  probability  table  (Table  28)  showing  the 
computed  probabilities  that  a  percentage  difference  as  large  as  the 
observed  difference  between  each  pair  of  groups,  in  each  category,  might 
have  arisen  by  chance.  Since  the  different  groups  do  not  include  the 
same  respondents,  the  percentages,  or  proportions,  are  independent,  and 
the  sampling  distribution  of  the  difference  between  two  proportions  is 
approximately  normal. 
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TABLE  28 


EXAMPLE  OF  PERZPROB  OUTPUT 

tASl  LOS  ANCitLES  SKILL  CtNlLH  GROUP  COMPARISONS  GOOD  THINGS  TO  DO 

numhek  of  Categories  =  lo 
NUMBER  of  groups  =  3 

data  will  rtt  RFAO  BY  COLUMNS 

GROUP  1  INTERVIEW  SURVEY  (N*15) 

GROUP  ?  WRITTEN  SURVEY  MALES  (Nsl5) 

GROUP  3  WRITTEN  SURVEY  FEMALES  (N*15) 

♦**  G  R  0  U  P  1  2  3 


number  of  SU 

B  J  E  C  T  S 

15 

15 

15 

category  index  and  T  I  t  l 

E 

«»«  p 

E  R  C  E 

N  T  / 

1 

SIUDY  and  LEAPM 

66*7 

53.3 

33.3 

2 

HE  A  GOOD  PEKSOF 

46.7 

33.3 

o 

. 

o 

3 

lEACH  and  guide  our  children 

46*7 

33.3 

53.3 

4 

HELP  CIHERS 

26.7 

40.0 

53.3 

5 

SUPPORT  yourself  nY  WORKING  aT  YOUR  TRADE 

66.7 

20.0 

o 

• 

o 

b 

HELP  The  POOR  PEOPLE 

33.3 

60.0 

33.3 

7 

HAVE  SOCIAL  PROGRESS 

6.7 

46.7 

20.0 

8 

improve  The  educational  system 

6.7 

40.0 

53.3 

9 

exercise  more  Control  over  young  people 

0.0 

26.7 

46.7 

10 

always  IRY  TO  IMPkOVF:  YOURSELFf  NOT  SLOW  DOWN  OR 

GIVE  UP 

33.3 

40.0 

26.7 

TW0-TAILE(^  PROhAf'ILlTY  UF  A  DIFFERENCE  OF  THIS  MAGNITUDE  ARISING  BY  CHANCE 


1/  2 

1/  3 

2/  3 

1 

.707 

.143 

.  4G  1 

2 

.707 

.245 

.681 

3 

.707 

.997 

.461 

4 

.700 

.285 

.716 

5 

.027 

.02  r 

1.000 

6 

.271 

1.000 

.271 

7 

.039 

.593 

.24  5 

8 

.085 

.017 

.716 

9 

.107 

.010 

•  449 

10 

.998 

.997 

.70  0 
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The  formula  for  computing  Z,  the  normal  deviate  (Wallis  and 
Roberts,  1956),  is; 


V(PQ)(l/n^  +  l/n2) 

where 

=  the  proportion  of  one  group  represented  in  the  category 

p^  =  the  proportion  of  other  group  represented  in  the  category 

n^  =  the  number  of  respondents  in  the  first  group 

n^  =  the  number  of  respondents  in  the  second  group 

n^p^  +  n2P2 
’’  '  ”l  +  “2 

Q  -  1  -  P 

This  formula  takes  the  null  hypothesis  seriously,  employing  an 
estimate  (P)  of  the  common  value  of  and  P^ ,  the  population  proportions. 

Since  the  possible  values  of  p^  -  P2  are  not  continuously  distrib¬ 
uted,  Yates’  correction  for  continuity  is  applied  before  the  probability 
is  computed,  reducing  by  0.5  the  frequency  corresponding  to  the  larger 
proportion  (n^^p^,  the  number  of  respondents  from  the  first  group  who  are 
represented  in  the  category)  and  increasing  by  0.5  the  frequency 
corresponding  to  the  smaller  proportion  (n2P2) •  This  correction  causes 
some  anomalous  but  trivial  fluctuations  in  cells  of  the  probability 
table  that  show  probabilities  of  differences  between  small  percentages 
of  small  samples. 

Each  value  in  the  probability  table  is  two  tailed,  including  the 
probability  of  the  difference  p^  >  P2  as  well  as  the  difference  P2  ^ 


155 


Additional  information  associated  with  the  percentage  table  includes: 
(1)  main  title  (for  example,  "East  Los  Angeles  Skill  Center,  Group 
Comparisons,  Good  Things  To  Do"),  (2)  number  of  categories  or  rows, 

(3)  number  of  groups  or  columns,  (4)  statement  of  how  the  data  were  input, 
that  is,  by  rows  or  columns,  (5)  a  one-line  description  of  each  group, 
and  (6)  the  number  of  subjects  (respondents)  in  each  group. 


Each  column  in  the  probability  table  compares  two  columns  (two 
groups)  in  the  percentage  table.  After  selecting  a  suitable  probability 
level  (for  example,  0.050,  0.010,  0.001),  the  user  can  circle  the  cells 
in  the  probability  table  that  meet  that  criterion.  The  circled  cells  then 
constitute  an  index  of  significant  differences  in  the  percentage  table. 


The  data  for  program  PERZPROB  are  the  percentages  printed  in  the 
second  UNIKOUNT  table,  percentages  of  groups  of  respondents.  Since  a 
different  UNIKOUNT  table  supplies  the  data  for  each  group,  and  since  the 
categories  are  unlikely  to  be  listed  in  the  same  order  in  any  two  UNIKOUNT 
tables,  a  table  of  these  percentages  is  made  up,  with  the  categories  and 
groups  assigned  to  table  rows  and  columns  as  desired.  The  PERZPROB  data 
cards  are  then  punched  from  this  table  either  by  rows  or  by  columns;  in 
the  usual  case,  punching  by  columns  is  more  economical. 


"'it  has  been  stated  that  the  group  percentages  are  independent  because 
the  different  groups  do  not  contain  the  same  respondents.  This  means 
that  any  entry  in  the  probability  table  refers  to  a  comparison  of  two 
independent  percentages.  A  word  of  caution  is  necessary,  however,  when 
the  user  circles  two  or  more  entries  in  the  same  column  of  the  proba¬ 
bility  table.  The  degree  of  dependence  of  entries  in  the  same  column 
is  indeterminate  but  should  not  be  assumed  to  be  zero.  Conditions 
tending  to  introduce  dependence  are:  (1)  each  respondent  may  be  repre¬ 
sented  in  several  cells  of  the  column,  (2)  each  respondent  is  allowed  a 
limited  number  of  responses,  which  may  be  distributed  across  a  larger 
number  of  cells,  and  (3)  the  cells  themselves  may  be  assumed  not  to 
exist  prior  to,  or  independent  of,  classification  of  the  responses  into 
ECHO  categories.  Because  of  this  intra-column  dependence,  the  user 
should  not  assume  a  simple  correspondence  between  the  number  of  cells 
circled  in  a  column  in  the  probability  table  and  the  degree  of  dissimi¬ 
larity  between  two  groups  of  respondents. 
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TABLE  29 


EXAMPLE  OF  ROCKEM  OUTPUT 


REFERENCE  COLUMNS  -  20-2K  COMPARISON  COLUMNS  -  23-24 


CATEGORY 

FREQUENCY 

RANK 

5 

8 

5.5 

CATEGORY 

FREQUENCY 

19 

2 

30 

2 

3 

1 

6 

1 

13 

1 

26 

1 

CATEGORY  TITLE 
TO  RESPECT  OTHERS  RIGHTS 

CATEGORY  TITLE 

TO  HAVE  PLEASANT  RELATIONSHIPS  WITH  OTHERS 
TO  BE  ON  TIME 

TO  CONSIDER  THE  VALUES  OF  OTHERS 
TO  ASSIST  OTHERS 
TO  BE  HONEST 

TO  DO  MORALLY  GOOD  THINGS 
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TABLE  30 


EXAMPLE  OF  ROCKEM  RESPONSE  LIST 


W  00804+ 

15  15 

TO 

W  00806+ 

15  24 

TO 

W  00807+ 

15  24 

TO 

W  01109+ 

15  09 

TO 

W  01304+ 

15  15 

TO 

W  01305+ 

15  09 

TO 

W  01409+ 

15  10 

TO 

W  01506+ 

15  10 

TO 

W  02001  + 

15  10 

TO 

W  02002+ 

15  09 

TO 

W  02003+ 

15  10 

TO 

W  02007+ 

15  24 

TO 

W  02010+ 

15  10 

TO 

W  02106+ 

15  24 

TO 

W  02201+ 

15  24 

TO 

W  02204+ 

15  15 

TO 

W  02205+ 

15  16 

TO 

W  02206+ 

15  15 

TO 

W  02208+ 

15  24 

TO 

W  02403+ 

15  15 

TO 

W  02404+ 

15  09 

TO 

W  02501+ 

15  10 

TO 

W  02503+ 

15  09 

TO 

W  02507+ 

15  10 

TO 

W  02509+ 

15  24 

TO 

W  02510+ 

15  15 

TO 

W  02703+ 

15  09 

TO 

W  02801+ 

15  10 

TO 

W  02804+ 

15  24 

TO 

W  02903+ 

15  10 

TO 

W  02905+ 

15  24 

TO 

W  00802+ 

07  16 

TO 

W  00803+ 

07  16 

TO 

LISTEN  TO  THE  RADIO 
GO  TO  THE  MOVIES 
WATCH  TV 

MAKE  A  TRIP  TO  THE  COUNTRY 

LISTEN  TO  MUSIC  WHILE  STUDYING 

GO  WITH  FATHER  AND  MOTHER  TO  THE  SEAS 

HAVE  ENOUGH  RELAXATION 

HAVE  FUN  BY  GOING  OUT  TO  THE  MOVIES 

WATCH  A  SHOOTING  MATCH  OR  TO  SHOOT 

MAKE  A  LONG  TRIP  TO  SEVERAL  PLACES 

LIE  DOWN  WHEN  ONE  HAS  NOTHING  TO  DO 

GO  TO  A  REALISTIC  MOVIE 

GO  OUT  FOR  FUN  WITH  CLOSE  FRIENDS 

GO  TO  ALL  KINDS  OF  MOVIES 

WATCH  TV 

LISTEN  TO  MUSIC 

READ  FICTION 

LISTEN  TO  NEWS  ON  THE  RADIO 
GO  TO  THE  MOVIES 

LISTEN  TO  MUSIC  WHICH  IS  NOT  TOO  FAST 

GO  FOR  A  PICNIC  OR  TO  TAKE  A  TRIP  FOR 

WATCH  GAMES 

MAKE  A  LONG  TRIP 

GO  OUT  FOR  FUN  WITH  FRIENDS 

60  TO  THE  MOVIES 

LISTEN  TO  MUSIC 

MAKE  A  TRIP  FOR  PLEASURE 

RELAX 

GO  TO  THE  MOVIES 
GO  OUT  FOR  FUN 
GO  TO  THE  MOVIES 
READ  ALL  KINDS  OF  BOOKS 
WRITE  A  BOOK 
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D,  PROGRAM  ROCKEM 

Program  ROCKEM  compares  two  classifications  of  the  same  responses, 
such  as  a  classification  on  Monday  by  three  males  and  another  classifica¬ 
tion  on  Wednesday  by  three  females.  The  program  can  also  list  the  data 
cards  sorted  by  category  number,  category  size,  or  both. 

The  program  facilitates  interpretation  of  any  category  from  one 
classification  (for  instance,  the  Monday  classification)  through  inspec¬ 
tion  of  the  distribution  of  the  cards  that  made  up  that  Monday  category 
across  one  or  more  categories  of  the  comparison  classification  (such  as 
the  Wednesday  classification) . 

The  user  specifies  two  data  fields  in  the  same  data  cards,  a 
"reference  field"  and  a  "comparison  field."  The  program  prints  a  small 
table  for  each  category  in  the  reference  classification  (Table  29) , 
identified  by  category  number  and  title,  and  showing  how  the  same  cards 
were  distributed  across  the  categories  of  the  comparison  classification, 
also  identified  by  category  number  and  title. 

The  user  can  also  have  all  of  the  data  cards  listed  in  reference- 
field  category-number  order,  or  by  decreasing  frequency  of  category 
number  (that  is,  largest  category  first).  Table  30  shows  31  responses 
in  category  15,  titled  "Relax  and  Go  Places  for  Fun,"  followed  by  two 
responses  from  the  next  largest  category,  category  7. 

E .  PROGRAM  PRINDEK 

Program  PRINDEK  prints  cards  for  reclassification,  one  value-and- 
source  response  per  card. 

The  purpose  of  having  reclassification  cards  is  to  facilitate  the 
classification  of  translated  responses  or  the  unbiased  combination  of 
disparate  samples  of  responses  (for  example,  where  differences  of 
vocabulary,  handwriting,  form  of  the  ECHO  question,  or  any  other  differ¬ 
ences  have  been  removed) . 
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The  main  input  for  program  PRINDEK  is  a  display  deck,  in  which 
each  response  has  been  punched  on  one  or  more  cards  (leaders  and 
trailers  ) .  The  most  important  output  of  program  PRINDEK  consists  of 
printed  reclassification  cards,  one  response  per  card.  Two  examples  are 
shown  in  Figure  7.  The  top  line  of  the  first  card  shows  a  nine-character 
identification  code,  printed  in  two  places.  (The  second  character  is  a 
blank  in  this  example  but  could  be  a  group  code.)  The  identification 
code  of  the  second  card  indicates  that  it  is  the  second  response  of  the 
same  respondent.  The  second  and  fourth  lines  on  each  card  contain  the 
ECHO  questions.  These  questions  are  input  to  the  program  and  can  be 
varied  by  the  user  as  need  be. 

F.  PROGRAM  NUDEK 

Program  NUDEK  merges  ECHO  reclassification  codes  with  previous 
codes  and  punched  responses.  NUDEK  can  also  rearrange  the  order  in  which 
information  is  punched  into  the  data  card  columns. 

The  main  use  for  program  NUDEK  is  to  update  an  ECHO  display  deck. 
Such  a  deck  contains  all  identification,  classification,  and  response 
information  up  to  a  given  time.  When  a  new  classification  of  the  same 
responses  is  performed,  using  either  reclassification  cards  or  the 
original  response  cards,  the  new  classification  codes  are  easily  punched 
into  the  newly  classified  cards,  but  it  is  less  convenient  to  punch 
them  into  the  display  deck.  However,  if  the  new  information  is  to  be 
fully  used,  it  must  be  combined  with  the  old.  Program  NUDEK  takes  the 
two  decks,  containing  the  old  and  new  information  respectively,  and 
produces  a  new  deck  that  contains  all  of  the  information.  At  the  same 
time,  if  desired,  the  arrangement  of  the  information  in  the  data  cards 
can  be  altered. 


See  Glossary  for  definitions  of  special  ECHO  terms. 
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Figure  7.  Examples  of  PRINDEK  Output  Cards 


When  the  new  display  deck  has  been  produced,  the  old  display  deck 
can  usually  be  discarded.  The  reclassification  cards  or  the  original 
response  cards  would  usually  be  retained  for  possible  further  use. 
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